Re: [Openstack] [RFC] OpenStack API

[Jorge Williams]

On Dec 30, 2010, at 3:43 PM, John Purrier wrote:

Hi Sandy, the easy one first… the OpenStack API is versioned, and so is 
backward compatible. The client will negotiate the highest version it knows 
about, the API returns will be consistent with that version.

Right.  We create new versions only when we introduce backward compatibility  
problems but maintain old versions and the ability to query version info to 
maintain compatibility.


As for the Easy API…. The on-going discussion is excellent and I believe we are 
flushing out some core issues. That being said, to date the plan is that there 
is a canonical OpenStack Compute API, and it is the one derived from the 
Rackspace Cloud Servers API.

I would like to have Jorge and others join the conversation so that everyone 
can see what the proposed OpenStack Compute v1.1 extension mechanisms are. As 
this spec is being finalized perhaps there is an opportunity to merge concepts, 
if it makes sense. It doesn’t make sense to have divergent efforts at this 
point IMO, I would like to have full traction on a complete and functional 
OpenStack API.


I have to admit I'm a little fuzzy on Easy API, but from reading the specs it 
seems that our ideas are highly compatible.  From my limited understanding it 
seems that Easy API may be the way by which you can make extensions work in 
OpenStack.  I'm looking forward to having more conversations on the subject as 
we move towards transitioning the API to OpenStack.


John

From: Sandy Walsh [mailto:sandy.wa...@rackspace.com]
Sent: Thursday, December 30, 2010 3:27 PM
To: John Purrier; 
openstack@lists.launchpad.net
Subject: RE: [Openstack] [RFC] OpenStack API

Thanks John. That's a good summary of the plan. I still have a few questions, 
if you could indulge me ...

Seems like there are two issues being debated, one customer-facing, one 
Nova-developer-facing:

Customer-facing:
Will OS 1.1 remain backward compatible with 1.0? In other words, is 1.1 truly a 
point release or a major version increase? A point release says to me it is 
backwards compatible.

Developer-facing:
The EasyAPI discussion (client-side tools *and* server-side implementation). 
The assumption here is that EasyAPI could/would be the OS 1.1 implementation. 
The answer to the customer-facing question will affect this debate.

Separately, there was also a discussion of the "hackability" of the OS API. I 
think we've seen that with the addition of pause/suspend/diagnostics and 
admin-only functionality that the current OS API is easily extensible. Are 
further discussions simply around implementation details? Are we debating a 
more data-driven implementation vs. a more explicit implementation? Are we 
debating replacing the current OS 1.0 implementation with EasyAPI? If so, would 
that be a practical use of resources given the current workload?

Sorry for the million questions, but I think we're getting close to the crux of 
the matter.

-S




From: 
openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
John Purrier [j...@openstack.org]
Sent: Thursday, December 30, 2010 3:59 PM
To: openstack@lists.launchpad.net
Subject: [Openstack] [RFC] OpenStack API
The timing of this is interesting, as it comes when there is a lot of 
discussion about EasyAPI… I would like to encourage discussions about the 
EasyAPI extensibility mechanisms versus the OpenStack API v1.1 extension 
mechanism. Jorge, you will need to let folks see the proposed updates to the 
OpenStack API for version 1.1.

It is clearly important to establish a viable OpenStack Compute API by the 
Bexar timeframe. We started the implementation in Austin based on the Rackspace 
Cloud Servers 1.0 API, now we need to finish the work. The Austin work is half 
finished since we could not create and promote the OpenStack namespace and 
still required eucatools to set up and manage a cluster. Presented for comment 
is an approach for handling the OpenStack API for Bexar and Cactus. 
Additionally, this sets a path for the handoff of the API specification from 
Rackspace to OpenStack and the Rackspace Cloud transition to the OpenStack API.

Where we are at:

a.   The implementation of the OpenStack API framework is complete, with 
the ability to return a “not implemented” indicator if the function is not 
available. This may occur due to differences in hypervisor support or due to 
work in progress.

b.   The current OpenStack API is version 1.0, this matches the version of 
the Rackspace Cloud Servers API in production in the cloud.

c.   Work needs to be done to setup and publish an OpenStack namespace. 
Additionally, the source will need to be updated to return the correct 
namespace URL.

d.   Work is in progress to

[Openstack] Glance/Nova Snapshot Changes

[Rick Harris]

In developing Nova's instance snapshots, I've run into a little snag
revolving around somed design decisions in both Nova and Glance.  I have a
plan that I'd like to move forward with ASAP, so please, if you see any
problems or have any objections, raise them now.

Problem
===

OpenStack's API divides calls into a "top-half" which returns quickly
(compute/api.py) and a fire-and-forget "bottom-half" (compute/manager.py)
which is long-running.

The problems is that the image-create OpenStack API call (which maps to
instance-snapshot) requires the image-metadata (id, name, status, etc) be
returned up front. The current implementation, however, doesn't create the
image in Glance until the snapshot is actually being uploaded which happens
well after the OpenStack API "top-half" has returned.

(* Aside: To facilitate caching, Glance treats image data as immutable, that
is one reason we wanted to hold off on creating the Image record in Glance
until the data was actually present. *)

Since we cannot change the OpenStack API (yet), we'll need to modify both Nova 
and
Glance to allow Images to exist *before* their data is available.

Proposed Solution
=

Here is my suggestion:

  * Glance images are given a status of 'queued', 'saving', 'active', or
'killed'. This field already exists-- I'm noting the statuses here because
we're going to start using them to enforce state. (Note: CloudServers has
a 'preparing' state which is no longer needed in the Nova-Glance
implementation)

  * We modify Glance to allow the client to omit the image data on
image-create (aka POST).  When Glance detects this, it creates a record
for the image in the registry, puts it into the 'queued' state, and then
returns the image metadata.

  * We modify Glance to allow data to be uploaded in a subsequent PUT request.
While the data is being uploaded, the image will be in the 'saving' state;
if the operation completes sucessfully, it will go to 'active', otherwise,
it will go to 'killed'. Note, since we have an immutability constraint on
images, we should not allow image data to be updated once it exists, so,
once the image goes 'active', subsequent PUT requests should fail with a
409 Conflict (or something similar). Also note, this is at odds somewhat
with ReSTful semantics since a PUT in this case (when image data is
present in the request body) is no longer idempotent. If we can think of
a better way, I'm all ears, however, for now I think the tradeoff is worth
it.

  * We modify OpenStack API image-create "top-half" to call
ImageService.create which will POST to Glance without the image data. We
will then return the image-metadata (with the image in a 'queued' state)
to the callee. The "top-half" will then pass the "image_id" to the
"bottom-half" which can upload the data into the specified image.

  * Modify Glance XenServer plugin to accept the image_id as an argument and
to PUT to that resource.


This is the bare-minimum that will need to change for now, down the road,
we'll need to consider:

  * Creating a monitor task that timeouts images that hang out in the 'queued'
and 'saving' state for too long, and a task that deletes images in the
'killed' status.

Also, Glance now has Python bindings in the form of its 'client.py'. As part
of this effort, I'm planning on modifying ImageService::Glance to use the new
Glance client. This will mean that Nova will require that Glance be installed
on the same machine running nova-api in order to use the Glance service.

Thoughts?


-Rick



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [RFC] OpenStack API

[Jorge Williams]

On Dec 30, 2010, at 1:59 PM, John Purrier wrote:

Jorge, you will need to let folks see the proposed updates to the OpenStack API 
for version 1.1.

Absolutely,  I will post the spec once we settle on the new set of API 
features.  The API extension mechanism and the transition to the OpenStack 
namespace are the major features in the release. Other features are minor from 
an API perspective (i.e. IPv6 address support).

We've been considering adding an extension mechanism to the Cloud Servers API  
even before OpenStack as we were approached by several third parties who wished 
to create their own implementations of the API.   The idea is that we want to 
allow folks to extend the API as they see fit while maintaing clear governance 
over the core spec and allowing for compatibility between different 
implementations.

I put together simple stack on API extensions, how they'd work, and how we'd 
recommend they be managed by OpenStack.  The API specs will flesh this out in 
more detail.  Some of the low level details may change slightly (namespaces, 
resource format, etc), but in general this is the approach that we're aiming 
for in the next revision of the spec.

http://wiki.openstack.org/JorgeWilliams?action=AttachFile&do=view&target=Extensions.pdf

As always, I welcome your feedback,

-jOrGe W.



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] [RFC] OpenStack Scope and projects

[Andy Smith]

http://wiki.openstack.org I would assume, under whatever names you think are
best, just so that there is a definitive doc that can be kept abreast of the
discussion.

On Thu, Dec 30, 2010 at 1:43 PM, John Purrier  wrote:

> I can create wiki pages, where should they live?
>
>
>
> John
>
>
>
> *From:* Andy Smith [mailto:andys...@gmail.com]
> *Sent:* Thursday, December 30, 2010 3:20 PM
> *To:* John Purrier
> *Cc:* openstack@lists.launchpad.net
> *Subject:* Re: [Openstack] [RFC] OpenStack Scope and projects
>
>
>
> I would quickly say this should be on the wiki so that it can be kept up to
> date as it is discussed, it is a document not an email.
>
> On Thu, Dec 30, 2010 at 12:59 PM, John Purrier  wrote:
>
> I would like to present for discussion a statement on the scope and
> projects that are considered core to OpenStack in the short term (2011).
> Additionally, this is a proposal to “formalize” how OpenStack includes and
> associates projects that are closely tied to the project but not part of the
> core.
>
>
>
> Why is this important to discuss? I believe it drives conversations about
> what to implement (priorities), how to implement (in Python in the core,
> through an extension mechanism, or via a published API). It is also
> important that the community is relatively on the same page about what
> OpenStack is, and what is outside the charter.
>
>
>
> Once the community has reached consensus the Project Oversight Board should
> review and publish as part of the overall project charter.
>
>
>
> Key concepts (proposed):
>
>
>
> OpenStack is scoped in the short term (ie. today and through 2011) to the
> following *core* components (note this is subject to modification by the
> Project Oversight Committee):
>
>
>
> · Cloud and Automation Infrastructure (Nova)
>
> ­   Virtual Machine Provisioning and Management (Compute)
>
> ­   Virtual Image Management and Tools (Glance)
>
> ­   Virtual Volume Provisioning and Management (Block Storage)
>
> ­   Virtual Network Provisioning and Management (Network)
>
>
>
> · Large Scale Data Storage and Processing (Swift)
>
> ­   Highly Available Object Storage
>
>
>
> The core projects are under project and governance control of the OpenStack
> open source project. These components will make up the OpenStack
> distribution packages that can be picked up by downstream distributions
> (such as Ubuntu). In order to ensure that there is ubiquitous distribution
> of the core OpenStack projects the development languages/environments will
> be limited to C, C++, and Python (other languages/runtimes that are
> ubiquitously available times might be considered by the POC in the future).
>
>
>
> OpenStack core projects will be presented as a series of services and will
> follow a common and agreed to service architecture. This will include:
>
>
>
> · A public RESTful API
>
> · A private RESTful management API
>
> · Optionally, a pub/sub notification interface
>
> · An extension interface to allow specific service behaviors to be
> plugged in
>
>
>
>
>
> Proprietary or open source modules may be plugged into the extension
> interface to provide deployment specific features and functionality. For
> example: OpenStack will provide a general interface for elastic block
> storage to the Compute nodes. A vendor, contractor, or hoster may plug a
> specific implementation of block storage under the service, i.e. proprietary
> interface to NetApp storage, and interface to Sheepdog block storage, etc.
>
>
>
> These extension modules have no defined OpenStack requirements, save they
> conform to the defined extension interface for the service. Language
> selection, distribution models, source license, etc. are all defined and
> controlled by the implementer(s).
>
>
>
> Note that the “public” service API’s are not necessarily exposed to
> OpenStack developers directly. For instance, the programming model for Nova
> will present a singular API endpoint, yet may expose Virtual Image or
> Virtual Volume operations. This aggregation of API functionality should be
> consistent across the OpenStack projects to allow a consistent programming
> model and, ultimately, is under the direction of the POC.
>
>
>
> In addition, there will be a concept of “affiliated” or “compatible”
> services for OpenStack that live outside of the core projects. These will
> generally be cloud services that extend the functionality of the base (core)
> OpenStack distribution. It is encouraged that these services be built using
> the same core service architectural concepts. Since these projects live
> outside of the core OpenStack projects they have the following
> characteristics:
>
>
>
> 1.   They are not subject to the OpenStack governance model or
> process. The governance will be the responsibility of the contributor(s).
>
> 2.   The responsibility for distribution will be on the
> contributor(s). These are optional OpenStack projects and may or may not be
> picke

Re: [Openstack] [RFC] OpenStack Scope and projects

[John Purrier]

I can create wiki pages, where should they live?

 

John

 

From: Andy Smith [mailto:andys...@gmail.com] 
Sent: Thursday, December 30, 2010 3:20 PM
To: John Purrier
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] [RFC] OpenStack Scope and projects

 

I would quickly say this should be on the wiki so that it can be kept up to 
date as it is discussed, it is a document not an email.

On Thu, Dec 30, 2010 at 12:59 PM, John Purrier  wrote:

I would like to present for discussion a statement on the scope and projects 
that are considered core to OpenStack in the short term (2011). Additionally, 
this is a proposal to “formalize” how OpenStack includes and associates 
projects that are closely tied to the project but not part of the core.

 

Why is this important to discuss? I believe it drives conversations about what 
to implement (priorities), how to implement (in Python in the core, through an 
extension mechanism, or via a published API). It is also important that the 
community is relatively on the same page about what OpenStack is, and what is 
outside the charter.

 

Once the community has reached consensus the Project Oversight Board should 
review and publish as part of the overall project charter.

 

Key concepts (proposed):

 

OpenStack is scoped in the short term (ie. today and through 2011) to the 
following core components (note this is subject to modification by the Project 
Oversight Committee):

 

· Cloud and Automation Infrastructure (Nova)

­   Virtual Machine Provisioning and Management (Compute)

­   Virtual Image Management and Tools (Glance)

­   Virtual Volume Provisioning and Management (Block Storage)

­   Virtual Network Provisioning and Management (Network)

 

· Large Scale Data Storage and Processing (Swift)

­   Highly Available Object Storage

 

The core projects are under project and governance control of the OpenStack 
open source project. These components will make up the OpenStack distribution 
packages that can be picked up by downstream distributions (such as Ubuntu). In 
order to ensure that there is ubiquitous distribution of the core OpenStack 
projects the development languages/environments will be limited to C, C++, and 
Python (other languages/runtimes that are ubiquitously available times might be 
considered by the POC in the future).

 

OpenStack core projects will be presented as a series of services and will 
follow a common and agreed to service architecture. This will include:

 

· A public RESTful API

· A private RESTful management API

· Optionally, a pub/sub notification interface

· An extension interface to allow specific service behaviors to be 
plugged in

 



 

Proprietary or open source modules may be plugged into the extension interface 
to provide deployment specific features and functionality. For example: 
OpenStack will provide a general interface for elastic block storage to the 
Compute nodes. A vendor, contractor, or hoster may plug a specific 
implementation of block storage under the service, i.e. proprietary interface 
to NetApp storage, and interface to Sheepdog block storage, etc.

 

These extension modules have no defined OpenStack requirements, save they 
conform to the defined extension interface for the service. Language selection, 
distribution models, source license, etc. are all defined and controlled by the 
implementer(s).

 

Note that the “public” service API’s are not necessarily exposed to OpenStack 
developers directly. For instance, the programming model for Nova will present 
a singular API endpoint, yet may expose Virtual Image or Virtual Volume 
operations. This aggregation of API functionality should be consistent across 
the OpenStack projects to allow a consistent programming model and, ultimately, 
is under the direction of the POC.

 

In addition, there will be a concept of “affiliated” or “compatible” services 
for OpenStack that live outside of the core projects. These will generally be 
cloud services that extend the functionality of the base (core) OpenStack 
distribution. It is encouraged that these services be built using the same core 
service architectural concepts. Since these projects live outside of the core 
OpenStack projects they have the following characteristics:

 

1.   They are not subject to the OpenStack governance model or process. The 
governance will be the responsibility of the contributor(s).

2.   The responsibility for distribution will be on the contributor(s). 
These are optional OpenStack projects and may or may not be picked up by the 
downstream distributions.

3.   OpenStack does not impose any language or runtime constraints on these 
services. The contributors need to weigh their runtime environment requirements 
against ease of development and desired ubiquity of distribution and deployment.

 

Examples of potential services include Database, Platform, Monitoring, etc. 
implementations.

 

A g

Re: [Openstack] [RFC] OpenStack API

[John Purrier]

Hi Sandy, the easy one first. the OpenStack API is versioned, and so is
backward compatible. The client will negotiate the highest version it knows
about, the API returns will be consistent with that version.

 

As for the Easy API.. The on-going discussion is excellent and I believe we
are flushing out some core issues. That being said, to date the plan is that
there is a canonical OpenStack Compute API, and it is the one derived from
the Rackspace Cloud Servers API. 

 

I would like to have Jorge and others join the conversation so that everyone
can see what the proposed OpenStack Compute v1.1 extension mechanisms are.
As this spec is being finalized perhaps there is an opportunity to merge
concepts, if it makes sense. It doesn't make sense to have divergent efforts
at this point IMO, I would like to have full traction on a complete and
functional OpenStack API.

 

John

 

From: Sandy Walsh [mailto:sandy.wa...@rackspace.com] 
Sent: Thursday, December 30, 2010 3:27 PM
To: John Purrier; openstack@lists.launchpad.net
Subject: RE: [Openstack] [RFC] OpenStack API

 

Thanks John. That's a good summary of the plan. I still have a few
questions, if you could indulge me ...

 

Seems like there are two issues being debated, one customer-facing, one
Nova-developer-facing:

 

Customer-facing:

Will OS 1.1 remain backward compatible with 1.0? In other words, is 1.1
truly a point release or a major version increase? A point release says to
me it is backwards compatible. 

 

Developer-facing:

The EasyAPI discussion (client-side tools *and* server-side implementation).
The assumption here is that EasyAPI could/would be the OS 1.1
implementation. The answer to the customer-facing question will affect this
debate. 

 

Separately, there was also a discussion of the "hackability" of the OS API.
I think we've seen that with the addition of pause/suspend/diagnostics and
admin-only functionality that the current OS API is easily extensible. Are
further discussions simply around implementation details? Are we debating a
more data-driven implementation vs. a more explicit implementation? Are we
debating replacing the current OS 1.0 implementation with EasyAPI? If so,
would that be a practical use of resources given the current workload?

 

Sorry for the million questions, but I think we're getting close to the crux
of the matter.

 

-S

 

 

 

  _  

From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf
of John Purrier [j...@openstack.org]
Sent: Thursday, December 30, 2010 3:59 PM
To: openstack@lists.launchpad.net
Subject: [Openstack] [RFC] OpenStack API

The timing of this is interesting, as it comes when there is a lot of
discussion about EasyAPI. I would like to encourage discussions about the
EasyAPI extensibility mechanisms versus the OpenStack API v1.1 extension
mechanism. Jorge, you will need to let folks see the proposed updates to the
OpenStack API for version 1.1.

 

It is clearly important to establish a viable OpenStack Compute API by the
Bexar timeframe. We started the implementation in Austin based on the
Rackspace Cloud Servers 1.0 API, now we need to finish the work. The Austin
work is half finished since we could not create and promote the OpenStack
namespace and still required eucatools to set up and manage a cluster.
Presented for comment is an approach for handling the OpenStack API for
Bexar and Cactus. Additionally, this sets a path for the handoff of the API
specification from Rackspace to OpenStack and the Rackspace Cloud transition
to the OpenStack API.

 

Where we are at:

 

a.   The implementation of the OpenStack API framework is complete, with
the ability to return a "not implemented" indicator if the function is not
available. This may occur due to differences in hypervisor support or due to
work in progress.

 

b.   The current OpenStack API is version 1.0, this matches the version
of the Rackspace Cloud Servers API in production in the cloud.

 

c.   Work needs to be done to setup and publish an OpenStack namespace.
Additionally, the source will need to be updated to return the correct
namespace URL. 

 

d.   Work is in progress to implement a set of tools (think eucatools
functionality) to allow Nova to be set up and administered through the
OpenStack API. Need to verify that this functionality will hit in Bexar.

 

The proposed plan:

 

1.   Bexar will present a version 1.0 OpenStack API based on the 1.0
Rackspace Cloud Servers API. The OpenStack namespace will be set up and
published, and tools will be available that manipulate Nova via the
OpenStack API. Any functionality that is not yet implemented will be
documented in the developer's guide.

 

2.   Rackspace will work to finalize the 1.1 Cloud Servers API spec.
Currently, this is thought of as the next version of the Rackspace Cloud
Servers API, but this should change to be considered the OpenStack 1

Re: [Openstack] [RFC] OpenStack API

[Sandy Walsh]

Thanks John. That's a good summary of the plan. I still have a few questions, 
if you could indulge me ...

Seems like there are two issues being debated, one customer-facing, one 
Nova-developer-facing:

Customer-facing:
Will OS 1.1 remain backward compatible with 1.0? In other words, is 1.1 truly a 
point release or a major version increase? A point release says to me it is 
backwards compatible.

Developer-facing:
The EasyAPI discussion (client-side tools *and* server-side implementation). 
The assumption here is that EasyAPI could/would be the OS 1.1 implementation. 
The answer to the customer-facing question will affect this debate.

Separately, there was also a discussion of the "hackability" of the OS API. I 
think we've seen that with the addition of pause/suspend/diagnostics and 
admin-only functionality that the current OS API is easily extensible. Are 
further discussions simply around implementation details? Are we debating a 
more data-driven implementation vs. a more explicit implementation? Are we 
debating replacing the current OS 1.0 implementation with EasyAPI? If so, would 
that be a practical use of resources given the current workload?

Sorry for the million questions, but I think we're getting close to the crux of 
the matter.

-S




From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
John Purrier [j...@openstack.org]
Sent: Thursday, December 30, 2010 3:59 PM
To: openstack@lists.launchpad.net
Subject: [Openstack] [RFC] OpenStack API


The timing of this is interesting, as it comes when there is a lot of 
discussion about EasyAPI… I would like to encourage discussions about the 
EasyAPI extensibility mechanisms versus the OpenStack API v1.1 extension 
mechanism. Jorge, you will need to let folks see the proposed updates to the 
OpenStack API for version 1.1.



It is clearly important to establish a viable OpenStack Compute API by the 
Bexar timeframe. We started the implementation in Austin based on the Rackspace 
Cloud Servers 1.0 API, now we need to finish the work. The Austin work is half 
finished since we could not create and promote the OpenStack namespace and 
still required eucatools to set up and manage a cluster. Presented for comment 
is an approach for handling the OpenStack API for Bexar and Cactus. 
Additionally, this sets a path for the handoff of the API specification from 
Rackspace to OpenStack and the Rackspace Cloud transition to the OpenStack API.



Where we are at:



a.   The implementation of the OpenStack API framework is complete, with 
the ability to return a “not implemented” indicator if the function is not 
available. This may occur due to differences in hypervisor support or due to 
work in progress.



b.   The current OpenStack API is version 1.0, this matches the version of 
the Rackspace Cloud Servers API in production in the cloud.



c.   Work needs to be done to setup and publish an OpenStack namespace. 
Additionally, the source will need to be updated to return the correct 
namespace URL.



d.   Work is in progress to implement a set of tools (think eucatools 
functionality) to allow Nova to be set up and administered through the 
OpenStack API. Need to verify that this functionality will hit in Bexar.



The proposed plan:



1.   Bexar will present a version 1.0 OpenStack API based on the 1.0 
Rackspace Cloud Servers API. The OpenStack namespace will be set up and 
published, and tools will be available that manipulate Nova via the OpenStack 
API. Any functionality that is not yet implemented will be documented in the 
developer’s guide.



2.   Rackspace will work to finalize the 1.1 Cloud Servers API spec. 
Currently, this is thought of as the next version of the Rackspace Cloud 
Servers API, but this should change to be considered the OpenStack 1.1 Compute 
API. If possible, this work to complete the spec should be complete by 
mid-January or so. This spec should assume an OpenStack namespace.



3.   With the publication of the 1.1 OpenStack Compute API spec, OpenStack 
should publish/highlight the provisions for API extensions. This will be the 
supported mechanism for projects to extend the official OpenStack API (without 
requiring version changes). Projects that are pushing to update/extend the 
OpenStack API will be directed to do it via the official extension mechanism 
and target the version 1.1 API. The version 1.0 OpenStack API will not be 
changed or extended.



4.   OpenStack will publish roadmap information stating that the 1.1 API 
will be available in April, via the Cactus release.



5.   Rackspace will implement the 1.1 OpenStack API spec and deploy support 
for it on the current Slicehost codebase. Support will be continued for the 1.0 
Rackspace API through version checking. At the point of deployment the 
Rackspace Cloud can claim OpenStack compatibility

Re: [Openstack] [RFC] OpenStack Scope and projects

[Andy Smith]

I would quickly say this should be on the wiki so that it can be kept up to
date as it is discussed, it is a document not an email.

On Thu, Dec 30, 2010 at 12:59 PM, John Purrier  wrote:

> I would like to present for discussion a statement on the scope and
> projects that are considered core to OpenStack in the short term (2011).
> Additionally, this is a proposal to “formalize” how OpenStack includes and
> associates projects that are closely tied to the project but not part of the
> core.
>
>
>
> Why is this important to discuss? I believe it drives conversations about
> what to implement (priorities), how to implement (in Python in the core,
> through an extension mechanism, or via a published API). It is also
> important that the community is relatively on the same page about what
> OpenStack is, and what is outside the charter.
>
>
>
> Once the community has reached consensus the Project Oversight Board should
> review and publish as part of the overall project charter.
>
>
>
> Key concepts (proposed):
>
>
>
> OpenStack is scoped in the short term (ie. today and through 2011) to the
> following *core* components (note this is subject to modification by the
> Project Oversight Committee):
>
>
>
> · Cloud and Automation Infrastructure (Nova)
>
> ­   Virtual Machine Provisioning and Management (Compute)
>
> ­   Virtual Image Management and Tools (Glance)
>
> ­   Virtual Volume Provisioning and Management (Block Storage)
>
> ­   Virtual Network Provisioning and Management (Network)
>
>
>
> · Large Scale Data Storage and Processing (Swift)
>
> ­   Highly Available Object Storage
>
>
>
> The core projects are under project and governance control of the OpenStack
> open source project. These components will make up the OpenStack
> distribution packages that can be picked up by downstream distributions
> (such as Ubuntu). In order to ensure that there is ubiquitous distribution
> of the core OpenStack projects the development languages/environments will
> be limited to C, C++, and Python (other languages/runtimes that are
> ubiquitously available times might be considered by the POC in the future).
>
>
>
> OpenStack core projects will be presented as a series of services and will
> follow a common and agreed to service architecture. This will include:
>
>
>
> · A public RESTful API
>
> · A private RESTful management API
>
> · Optionally, a pub/sub notification interface
>
> · An extension interface to allow specific service behaviors to be
> plugged in
>
>
>
>
>
> Proprietary or open source modules may be plugged into the extension
> interface to provide deployment specific features and functionality. For
> example: OpenStack will provide a general interface for elastic block
> storage to the Compute nodes. A vendor, contractor, or hoster may plug a
> specific implementation of block storage under the service, i.e. proprietary
> interface to NetApp storage, and interface to Sheepdog block storage, etc.
>
>
>
> These extension modules have no defined OpenStack requirements, save they
> conform to the defined extension interface for the service. Language
> selection, distribution models, source license, etc. are all defined and
> controlled by the implementer(s).
>
>
>
> Note that the “public” service API’s are not necessarily exposed to
> OpenStack developers directly. For instance, the programming model for Nova
> will present a singular API endpoint, yet may expose Virtual Image or
> Virtual Volume operations. This aggregation of API functionality should be
> consistent across the OpenStack projects to allow a consistent programming
> model and, ultimately, is under the direction of the POC.
>
>
>
> In addition, there will be a concept of “affiliated” or “compatible”
> services for OpenStack that live outside of the core projects. These will
> generally be cloud services that extend the functionality of the base (core)
> OpenStack distribution. It is encouraged that these services be built using
> the same core service architectural concepts. Since these projects live
> outside of the core OpenStack projects they have the following
> characteristics:
>
>
>
> 1.   They are not subject to the OpenStack governance model or
> process. The governance will be the responsibility of the contributor(s).
>
> 2.   The responsibility for distribution will be on the
> contributor(s). These are optional OpenStack projects and may or may not be
> picked up by the downstream distributions.
>
> 3.   OpenStack does not impose any language or runtime constraints on
> these services. The contributors need to weigh their runtime environment
> requirements against ease of development and desired ubiquity of
> distribution and deployment.
>
>
>
> Examples of potential services include Database, Platform, Monitoring, etc.
> implementations.
>
>
>
> A graphical view:
>
>
>
>
>
>
>
>
>
> ___
> Mailing list: https://launchpad.net/

[Openstack] [RFC] OpenStack Image Formats

[John Purrier]

A discussion in order to set a common understanding of supported image
formats in OpenStack and how the project can approach the issues surrounding
various hypervisors, cross-cloud interoperability, and potentially setting
some necessary development work items. Once the community has reached
consensus the Project Oversight Committee should consider and set a clear
project set of guidelines.

 

Image Formats and workload portability in OpenStack and public and/or
private cloud deployments.

 

The key point of the discussion: Virtual Image Formats matter. As the longer
term vision for OpenStack Compute and the cloud computing industry is
formulated it is becoming clear that over the next few years the issues of
portable workloads, federated cloud deployments, and seamless operations
across multi-hypervisor systems (whether they are inside a single DC or
federated across two or more deployments) will be a key element of success.
OpenStack Compute will form the technological basis for achieving this.

 

However, stating that Image Formats matter does not imply that there will be
a "winner" in the marketplace or that a singular format is pushed via the
OpenStack project. In order to achieve the goals stated above surrounding
workload portability the key element will be not the actual virtual disk
formats; but rather a standard VM interchange standard and the ability to
easily re-purpose the VM image to the target environment (whether it be a
public cloud or a private instance).

 

Today we are seeing most, if not all, of the popular virtualization systems
branching to support alternate VM image formats beyond what they consider
"native", either through conversion tools or the ability to natively mount
and run the alternative formats. This demonstrates that the market
understands the importance of being able to be somewhat image format
agnostic and that there is no value in trying to lock-in customers through
proprietary formats.

 

The Virtual Disk Format playing field today looks like:

 

. VHD - Citrix XenServer and Microsoft Hyper-V

. VMDK - VMWare ESX

. VDI - Oracle VirtualBox

. QCOW2 - KVM and QEMU

 

All the hypervisors also support Raw Disk Images. Additionally, deployments
can combine a disk partitioning and file-system (usually via LVM) to get
more direct control of Raw Images and to provide some additional features,
such as Snapshot with Coalesce (as Rackspace does on the current Linux Cloud
Servers product).

 

These disk formats are not too complicated, and there are free and cheap
tools that allow conversions between the various formats. As stated above,
there is a movement within all of these projects to reduce the friction in
moving VM images into their environment that were created by someone else.

 

Proposal for comment:

 

OpenStack has, at its core, a mission to provide widespread ubiquity of
Virtual/Cloud Computing and also has taken a hypervisor-agnostic approach.
In order to foster this and to move the project to a world where workloads
can be easily moved between installations that are running different
underlying virtualization systems the following should occur:

 

1.  A standardized VM Image exchange format should be described. The
DMTF already has a specification that is useful here, OVF. OVF is a
specification that provides for an exchange container that will include
standardized mechanisms to describe meta-data and also to encapsulate one or
more VM images or .ISO files. OVF does not dictate any particular VM Image
format, and hence is ideal for our purposes.

 

An OpenStack VM interchange specification should be drafted and reviewed by
the OpenStack community.

 

2.  OpenStack should *not* specify a "preferred" or "default" virtual
disk format. Just as a core tenet is hypervisor agnosticism another key
position will be VM Image agnosticism. The message is that each deployment
should choose the best hypervisor and image format for their particular
goals. This allows companies like Rackspace or others to choose the systems
and formats that best support the features they want to take to the market.

 

This also sidesteps the potentially divisive topic of support for VHD and
the Microsoft Open Promise licensing. Since support will be optional for all
formats if someone is adamant about not supporting VHD in their OpenStack
deployment they are free to not include it. Others, including Rackspace, may
choose to support VHD for the features it provides.

 

3.  The OpenStack Image Repository project ("Glance") should be extended
to provide native image conversion capabilities. This will hide the
differences in formats from the deployment and operational decisions and
allow workloads to easily be imported and exported between both OpenStack
installations, and also to/from native virtualization installations (such as
VMware).

 

A blueprint and initial vision specification should be drafted and submitted
to the OpenStack Launchpa

[Openstack] [RFC] OpenStack API

[John Purrier]

The timing of this is interesting, as it comes when there is a lot of
discussion about EasyAPI. I would like to encourage discussions about the
EasyAPI extensibility mechanisms versus the OpenStack API v1.1 extension
mechanism. Jorge, you will need to let folks see the proposed updates to the
OpenStack API for version 1.1.

 

It is clearly important to establish a viable OpenStack Compute API by the
Bexar timeframe. We started the implementation in Austin based on the
Rackspace Cloud Servers 1.0 API, now we need to finish the work. The Austin
work is half finished since we could not create and promote the OpenStack
namespace and still required eucatools to set up and manage a cluster.
Presented for comment is an approach for handling the OpenStack API for
Bexar and Cactus. Additionally, this sets a path for the handoff of the API
specification from Rackspace to OpenStack and the Rackspace Cloud transition
to the OpenStack API.

 

Where we are at:

 

a.   The implementation of the OpenStack API framework is complete, with
the ability to return a "not implemented" indicator if the function is not
available. This may occur due to differences in hypervisor support or due to
work in progress.

 

b.   The current OpenStack API is version 1.0, this matches the version
of the Rackspace Cloud Servers API in production in the cloud.

 

c.   Work needs to be done to setup and publish an OpenStack namespace.
Additionally, the source will need to be updated to return the correct
namespace URL. 

 

d.   Work is in progress to implement a set of tools (think eucatools
functionality) to allow Nova to be set up and administered through the
OpenStack API. Need to verify that this functionality will hit in Bexar.

 

The proposed plan:

 

1.   Bexar will present a version 1.0 OpenStack API based on the 1.0
Rackspace Cloud Servers API. The OpenStack namespace will be set up and
published, and tools will be available that manipulate Nova via the
OpenStack API. Any functionality that is not yet implemented will be
documented in the developer's guide.

 

2.   Rackspace will work to finalize the 1.1 Cloud Servers API spec.
Currently, this is thought of as the next version of the Rackspace Cloud
Servers API, but this should change to be considered the OpenStack 1.1
Compute API. If possible, this work to complete the spec should be complete
by mid-January or so. This spec should assume an OpenStack namespace.

 

3.   With the publication of the 1.1 OpenStack Compute API spec,
OpenStack should publish/highlight the provisions for API extensions. This
will be the supported mechanism for projects to extend the official
OpenStack API (without requiring version changes). Projects that are pushing
to update/extend the OpenStack API will be directed to do it via the
official extension mechanism and target the version 1.1 API. The version 1.0
OpenStack API will not be changed or extended.

 

4.   OpenStack will publish roadmap information stating that the 1.1 API
will be available in April, via the Cactus release.

 

5.   Rackspace will implement the 1.1 OpenStack API spec and deploy
support for it on the current Slicehost codebase. Support will be continued
for the 1.0 Rackspace API through version checking. At the point of
deployment the Rackspace Cloud can claim OpenStack compatibility; timing is
controlled by Rackspace product. Any client development against the Cloud
Servers infrastructure that is 1.1 compatible will be able to run unchanged
once Rackspace transitions from the Slicehost codebase to the Nova codebase.

 

6.   Once the version 1.1 specification is complete, the "ownership" of
the spec will be transferred to OpenStack. Practically, Rackspace will
continue to drive innovation through versioning and extensions, but the
official spec will be under the direction of the OpenStack project and
future contributions from Rackspace and others will follow the standard,
open processes. Rackspace and the community will be able to leverage the
published OpenStack developer's guides and other documentation for having
developers interact with their OpenStack Nova deployments.

 

Comments? 

 

Rick/Thierry, if we have work items for Bexar that are not covered in order
to complete this plan let's highlight them ASAP. Also, can we verify that
the management tools are in progress?

 

John

 

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Easy API

[Sandy Walsh]

Thanks Andy ... that answers many of our questions.

I haven't had an opportunity to look at Easy API yet, but I don't think there 
is any reason to break compatibility with the RS/OS API. Here's how I see the 
flow ('Client' is the client-side command line tool):

1. Client determines Verb of Interest from command line ('create-instance', 
'pause', 'add-ip', etc)
2. Client polls the server for WADL/JSON to get that Verb's expected parameter 
list, URL template and Posting Method.
3. Client dynamically builds gflag/argparse configuration based on parameter 
list
4. Remaining cmdline arguments are parsed and data types checked based by 
argparse
5. The Verb's URL template is populated with arguments and submitted via 
Posting Method (POST, GET, etc)
6. Results obtained ... Profit

Since the URL template came from the server as well as the Posting method it 
should work fine with RS API. Even the little idiosyncratic quirks can be 
handled.

If this is the approach easy-api is taking ... sounds like a great replacement 
for python-cloudservers. Although I'm not convinced introspective approaches 
won't require hacky tweaks, thus defeating the effort.

Certainly adding Yet Another API doesn't seem like a good idea given the 
business advantages of maintaining compatibility with existing 
clients/partners. Extending the current API offers a smoother migration path 
for all.

I'm sure I'm missing something or just crazy in these assumptions.

Look forward to your feedback ... in the meanwhile I'll look at easy-api.

Cheers,
Sandy




From: Andy Smith [andys...@gmail.com]
Sent: Wednesday, December 29, 2010 4:08 PM
To: Matt Dietz
Cc: Jesse Andrews; termie; Sandy Walsh; openstack@lists.launchpad.net
Subject: Re: Easy API

+openstack

On Wed, Dec 29, 2010 at 12:06 PM, Andy Smith 
mailto:andys...@gmail.com>> wrote:
Heya Matt,

I was intending on writing an email to the mailing list about it (so I'll just 
CC it), was just going over tone with the Anso team first since I tend to come 
off a bit antagonistic.

The short summary is that we wanted to return "hackability" to the project, as 
developers the move towards API compatibility has slowly removed most of the 
tools we had in place to test out new features as they were being worked on, so 
in that way the reflection api, self-registration of services, command-line 
tool and general simplicity (you can explain how everything works in just a few 
sentences) are a solution.

Part of that hackability also meant hackability for third-parties, we wanted 
developers to be able to add functionality to Nova without having to go through 
the Nova review process and even be able package that functionality separately, 
this is an obvious  goal for any large entity using openstack internally where 
they have additional changes to make to customize their deployment.

Easy API makes up a good basis for extensibility of the project over time.

The existing APIs are still necessary to serve their purpose which is 
translating the intents defined by existing specifications/implementations to 
tasks that make sense for Nova, and should it be desirable those interfaces can 
easily be built on top of and decoupled (due to the delegated auth pattern) 
from the Easy API, the EasyCloudTestCase gives a decent example of how such a 
thing could be done.

In my personal opinion I see the two existing API implementations as 
'compatibility' APIs where as Easy API is a direct API and much easier to 
understand because of it.

The relevant blueprint (read the spec, obvs): 
https://blueprints.launchpad.net/nova/+spec/easy-api

--andy

On Wed, Dec 29, 2010 at 11:20 AM, Matt Dietz 
mailto:matt.di...@rackspace.com>> wrote:
Hey guys,

I was wondering if you could answer a few questions? I get the argument that 
EC2 and eucatools suck, and that the OS API is incomplete/unstable. What 
prompted you starting down the Easy API path? Subsequently, what issues are you 
hoping to solve with it? From what I can see, it's the WADL-like/reflection 
interface that seems to be the driving force. Is this correct?
I'm curious mostly because a lot of effort has been expended with the goal of 
making the existing OS API the canonical one. I'm fine with a better effort, 
but I've been operating in accordance with the internal goal of making it work 
100% like the Rackspace API so all existing bindings will "just work" if 
someone cares to try out Openstack. Obviously we can't continue working on 
parallel paths because we'll just end up fracturing the  API user-base until we 
can decide which API is going to stick.
I'd like to get the discussion started, and I'll be glad to take it to the 
blueprint (since I see you made one as of 18 hours ago) or IRC.

Thanks,

Matt


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.laun

Re: [Openstack] Easy API

[Matt Dietz]

Thanks for the reply,

I have to confess that I'm still concerned about the idea of yet another API 
for several reasons.

* While I appreciate the idea of a compatibility layer of sorts, we already 
have a weird digression in available features between Amazon and Openstack. I 
think a 3rd API that also isn't finished will lead to more confusion than 
anything else, especially since the Openstack API is named to imply that it's 
official.
* Couldn't we just add a reflection API to the OS API and gain many of the same 
benefits you list?
* Stack doesn't seem to solve the binding problem for people that want to 
programatically interact with Openstack. As such, this means more API bindings, 
and I don't think reflection helps with much of anything in that regard. More 
specifically, said bindings will still either be written with explicit needs in 
mind or as a crack at being wholly comprehensive. 
http://bitworking.org/news/193/Do-we-need-WADL articulates this better than I 
can.
* Presumably EasyAPI will also be incomplete/unstable through at least Cactus. 
Doesn't this just add to the problem that drove you to create EasyAPI in the 
first place?
* I think the hackability idea is sound, with the caveat that I don't really 
see how this implementation actually solves that issue any better than the 
existing code base. With a small amount of effort, we can extend the OS API to 
be highly modular.

I do want to mention that at no point was the OS API, in it's current 
incarnation, intended to be the do-all/be-all. We're simply making a first pass 
for parity, and then follow-up passes to make it awesome :-)

Matt

From: Andy Smith mailto:andys...@gmail.com>>
Date: Wed, 29 Dec 2010 12:08:29 -0800
To: Matt Dietz mailto:matt.di...@rackspace.com>>
Cc: Jesse Andrews mailto:je...@ansolabs.com>>, termie 
mailto:launch...@anarkystic.com>>, Sandy Walsh 
mailto:sandy.wa...@rackspace.com>>, 
mailto:openstack@lists.launchpad.net>>
Subject: Re: Easy API

+openstack

On Wed, Dec 29, 2010 at 12:06 PM, Andy Smith 
mailto:andys...@gmail.com>> wrote:
Heya Matt,

I was intending on writing an email to the mailing list about it (so I'll just 
CC it), was just going over tone with the Anso team first since I tend to come 
off a bit antagonistic.

The short summary is that we wanted to return "hackability" to the project, as 
developers the move towards API compatibility has slowly removed most of the 
tools we had in place to test out new features as they were being worked on, so 
in that way the reflection api, self-registration of services, command-line 
tool and general simplicity (you can explain how everything works in just a few 
sentences) are a solution.

Part of that hackability also meant hackability for third-parties, we wanted 
developers to be able to add functionality to Nova without having to go through 
the Nova review process and even be able package that functionality separately, 
this is an obvious  goal for any large entity using openstack internally where 
they have additional changes to make to customize their deployment.

Easy API makes up a good basis for extensibility of the project over time.

The existing APIs are still necessary to serve their purpose which is 
translating the intents defined by existing specifications/implementations to 
tasks that make sense for Nova, and should it be desirable those interfaces can 
easily be built on top of and decoupled (due to the delegated auth pattern) 
from the Easy API, the EasyCloudTestCase gives a decent example of how such a 
thing could be done.

In my personal opinion I see the two existing API implementations as 
'compatibility' APIs where as Easy API is a direct API and much easier to 
understand because of it.

The relevant blueprint (read the spec, obvs): 
https://blueprints.launchpad.net/nova/+spec/easy-api

--andy

On Wed, Dec 29, 2010 at 11:20 AM, Matt Dietz 
mailto:matt.di...@rackspace.com>> wrote:
Hey guys,

I was wondering if you could answer a few questions? I get the argument that 
EC2 and eucatools suck, and that the OS API is incomplete/unstable. What 
prompted you starting down the Easy API path? Subsequently, what issues are you 
hoping to solve with it? From what I can see, it's the WADL-like/reflection 
interface that seems to be the driving force. Is this correct?
I'm curious mostly because a lot of effort has been expended with the goal of 
making the existing OS API the canonical one. I'm fine with a better effort, 
but I've been operating in accordance with the internal goal of making it work 
100% like the Rackspace API so all existing bindings will "just work" if 
someone cares to try out Openstack. Obviously we can't continue working on 
parallel paths because we'll just end up fracturing the  API user-base until we 
can decide which API is going to stick.
I'd like to get the discussion started, and I'll be glad to take it to the 
blueprint (since I see you made one as of 18 hours ago) or IRC.

Thanks,

Matt




Confidentiality No

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Erik Carlin]

Agree the scale limits will increase over time. I'm thinking more about
what the near term (cactus or cactus+1) limits should be.

The model we have been using is a single nova deployment per metro area
(called a region).  That metro area could be comprised of multiple Dcs and
failure isolation domains (Amazon calls these availablity zones) within
those Dcs.  I recognize that we've been trying to have a flexible model
here and not dictate a specific deployment architecture, but I also think
there are some advantages in not trying to be everything to everyone.
Maybe we should come up with several common deployment scenarios at
different scale, standardize nomenclature, and try to work towards those??

Erik

On 12/30/10 12:38 PM, "Bret Piatt"  wrote:

>These numbers seem very large today but a decade from now they won't at
>all.
>We don't need to design everything for the requirements in 2020 but we
>should have some view on where things are going.  More swag
>calculations...
>
>In 2020...
>60,000,000 servers worldwide (I'm assuming this doesn't grow super quickly
>as instead of more, we're making more powerful with additional cores/RAM
>per
>server)
>50% of those servers are in "cloud" so 30,000,000
>I'll call a "major cloud player" somebody with 10% market share, so
>3,000,000 servers
>For geographic spread I'll assume 10-15 facilities so 200,000-300,000
>servers per facility
>
>Each facility may be broken up into zones so the question is at what point
>do we make the cross-over from intra-cloud to inter-cloud?  If it is at
>the
>zone level we're looking at 20-50k servers per zone with increasing VM
>density so 50-100 VMs per server for a total of 1 million to 5 million
>VMs.
>At each point in the architecture where the cost model changes for
>inter-VM
>traffic we need to have a logical construct in place.  Potentially
>zone->facility->region->world is enough granularity, we may need
>zone->facility->metro area->region->world.  Where we switch from
>intra-cloud
>(singular management domain) to inter-cloud (connected through a cloud
>directory service with federation) needs to take into account the overhead
>for brokering the additional complexity of inter-cloud communication
>versus
>the overhead of a larger singular management domain.
>
>Bret Piatt
>OpenStack.org
>Twitter: @bpiatt / Mobile 210-867-9338
>Open Source Cloud Infrastructure: http://www.openstack.org
>
>
>
>-Original Message-
>From: openstack-bounces+bret=openstack@lists.launchpad.net
>[mailto:openstack-bounces+bret=openstack@lists.launchpad.net] On
>Behalf
>Of Erik Carlin
>Sent: Thursday, December 30, 2010 12:15 PM
>To: Rick Clark
>Cc: openstack@lists.launchpad.net
>Subject: Re: [Openstack] Some insight into the number of instances Nova
>needs to spin up...
>
>I suggest we consider the limits of a single nova deployment, not across
>all
>regions.  To Pete's point, at a certain scale, people will break into
>parallel, independent nova deployments.  A single, global, deployment
>becomes untenable at scale.
>
>I agree with Pete that 1M hosts (and 45M Vms) is a bit out of whack for a
>single nova deployment.  As a frame of reference, here are a couple of
>links
>that estimate total server count by the big boys:
>
>http://www.datacenterknowledge.com/archives/2009/05/14/whos-got-the-most-w
>e
>b-servers/
>http://www.intac.net/a-comparison-of-dedicated-servers-by-company_2010-04-
>1
>3/
>
>Google is the largest and they are estimated to run ~1M+ servers.
>Microsoft is ~500K+.  Facebook is around 60K.  This link
>(http://royal.pingdom.com/2009/08/24/google-may-own-more-than-2-of-all-ser
>v
>ers-in-the-world/) is a few years old and puts the total worldwide server
>count at 44M.
>
>I submit that setting the nova limit to match Google's total server count
>and 1/44th of the total worldwide server count is overkill.
>
>The limits I suggest below are not per AZ, but per nova deployment (there
>could be multiple AZs inside of a deployment).  I think we may need to
>clarify nomenclature (although it may just be me since I haven't been too
>engaged in these discussions to date).  I know at the last design summit
>it
>was decided to call everything a "zone".
>
>Erik
> 
>
>On 12/30/10 11:42 AM, "Rick Clark"  wrote:
>
>>Actually it was 1M hosts and  Ivthink 45 million vms.  It was meant to
>>be across all regions.  Jason Seats set the number arbitrarily, but it
>>is a good target to not let us forget about scaling while we design.
>>
>>I think eventually all loads will be more ephemeral.  So, I think I
>>agree with your numbers, if you are talking about a single availability
>>zone.
>>
>>On 12/30/2010 11:25 AM, Erik Carlin wrote:
>>> You are right.  The 1M number was VMs not hosts.  At least, that was
>>>from  one scale discussion we had within Rackspace.  I'm not sure what
>>>the  "official" nova target limits are and I can't find anything on
>>>launchpad  that defines it.  If there is something, could someone
>>>please send me a  link.
>>> 
>>

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Bret Piatt]

These numbers seem very large today but a decade from now they won't at all.
We don't need to design everything for the requirements in 2020 but we
should have some view on where things are going.  More swag calculations...

In 2020...
60,000,000 servers worldwide (I'm assuming this doesn't grow super quickly
as instead of more, we're making more powerful with additional cores/RAM per
server)
50% of those servers are in "cloud" so 30,000,000
I'll call a "major cloud player" somebody with 10% market share, so
3,000,000 servers
For geographic spread I'll assume 10-15 facilities so 200,000-300,000
servers per facility

Each facility may be broken up into zones so the question is at what point
do we make the cross-over from intra-cloud to inter-cloud?  If it is at the
zone level we're looking at 20-50k servers per zone with increasing VM
density so 50-100 VMs per server for a total of 1 million to 5 million VMs.
At each point in the architecture where the cost model changes for inter-VM
traffic we need to have a logical construct in place.  Potentially
zone->facility->region->world is enough granularity, we may need
zone->facility->metro area->region->world.  Where we switch from intra-cloud
(singular management domain) to inter-cloud (connected through a cloud
directory service with federation) needs to take into account the overhead
for brokering the additional complexity of inter-cloud communication versus
the overhead of a larger singular management domain.

Bret Piatt
OpenStack.org
Twitter: @bpiatt / Mobile 210-867-9338
Open Source Cloud Infrastructure: http://www.openstack.org



-Original Message-
From: openstack-bounces+bret=openstack@lists.launchpad.net
[mailto:openstack-bounces+bret=openstack@lists.launchpad.net] On Behalf
Of Erik Carlin
Sent: Thursday, December 30, 2010 12:15 PM
To: Rick Clark
Cc: openstack@lists.launchpad.net
Subject: Re: [Openstack] Some insight into the number of instances Nova
needs to spin up...

I suggest we consider the limits of a single nova deployment, not across all
regions.  To Pete's point, at a certain scale, people will break into
parallel, independent nova deployments.  A single, global, deployment
becomes untenable at scale.

I agree with Pete that 1M hosts (and 45M Vms) is a bit out of whack for a
single nova deployment.  As a frame of reference, here are a couple of links
that estimate total server count by the big boys:

http://www.datacenterknowledge.com/archives/2009/05/14/whos-got-the-most-we
b-servers/
http://www.intac.net/a-comparison-of-dedicated-servers-by-company_2010-04-1
3/

Google is the largest and they are estimated to run ~1M+ servers.
Microsoft is ~500K+.  Facebook is around 60K.  This link
(http://royal.pingdom.com/2009/08/24/google-may-own-more-than-2-of-all-serv
ers-in-the-world/) is a few years old and puts the total worldwide server
count at 44M.

I submit that setting the nova limit to match Google's total server count
and 1/44th of the total worldwide server count is overkill.

The limits I suggest below are not per AZ, but per nova deployment (there
could be multiple AZs inside of a deployment).  I think we may need to
clarify nomenclature (although it may just be me since I haven't been too
engaged in these discussions to date).  I know at the last design summit it
was decided to call everything a "zone".

Erik
 

On 12/30/10 11:42 AM, "Rick Clark"  wrote:

>Actually it was 1M hosts and  Ivthink 45 million vms.  It was meant to 
>be across all regions.  Jason Seats set the number arbitrarily, but it 
>is a good target to not let us forget about scaling while we design.
>
>I think eventually all loads will be more ephemeral.  So, I think I 
>agree with your numbers, if you are talking about a single availability 
>zone.
>
>On 12/30/2010 11:25 AM, Erik Carlin wrote:
>> You are right.  The 1M number was VMs not hosts.  At least, that was 
>>from  one scale discussion we had within Rackspace.  I'm not sure what 
>>the  "official" nova target limits are and I can't find anything on 
>>launchpad  that defines it.  If there is something, could someone 
>>please send me a  link.
>> 
>> I'm am certain that Google can manage more than 10K physical servers 
>> per DC. Rackspace does this today.
>> 
>> If I combine what I know about EC2 and Cloud Servers, I would set the 
>>ROM  scale targets as:
>> 
>> ABSOLUTE
>> 1M VMs
>> 50K hosts
>> 
>> RATE
>> 500K transactions/day (create/delete server, list servers, resize 
>>server,  etc. - granted, some are more expensive than others)  That 
>>works out to ~21K/hr but it won't be evenly distributed.  To allow  
>>for peak, I would say something like 75K/hour or ~21/sec.
>> 
>> Thoughts?
>> 
>> Erik
>> 
>> 
>> On 12/30/10 9:20 AM, "Pete Zaitcev"  wrote:
>> 
>>> On Wed, 29 Dec 2010 19:27:09 +
>>> Erik Carlin  wrote:
>>>
 The 1M host limit still seems reasonable to me. []
>>>
>>> In my opinion, such numbers are completely out of whack. Google's 
>>>Chubby  article says that the busiest C

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Erik Carlin]

I suggest we consider the limits of a single nova deployment, not across
all regions.  To Pete's point, at a certain scale, people will break into
parallel, independent nova deployments.  A single, global, deployment
becomes untenable at scale.

I agree with Pete that 1M hosts (and 45M Vms) is a bit out of whack for a
single nova deployment.  As a frame of reference, here are a couple of
links that estimate total server count by the big boys:

http://www.datacenterknowledge.com/archives/2009/05/14/whos-got-the-most-we
b-servers/
http://www.intac.net/a-comparison-of-dedicated-servers-by-company_2010-04-1
3/

Google is the largest and they are estimated to run ~1M+ servers.
Microsoft is ~500K+.  Facebook is around 60K.  This link
(http://royal.pingdom.com/2009/08/24/google-may-own-more-than-2-of-all-serv
ers-in-the-world/) is a few years old and puts the total worldwide server
count at 44M.

I submit that setting the nova limit to match Google's total server count
and 1/44th of the total worldwide server count is overkill.

The limits I suggest below are not per AZ, but per nova deployment (there
could be multiple AZs inside of a deployment).  I think we may need to
clarify nomenclature (although it may just be me since I haven't been too
engaged in these discussions to date).  I know at the last design summit
it was decided to call everything a "zone".

Erik
 

On 12/30/10 11:42 AM, "Rick Clark"  wrote:

>Actually it was 1M hosts and  Ivthink 45 million vms.  It was meant to
>be across all regions.  Jason Seats set the number arbitrarily, but it
>is a good target to not let us forget about scaling while we design.
>
>I think eventually all loads will be more ephemeral.  So, I think I
>agree with your numbers, if you are talking about a single availability
>zone.
>
>On 12/30/2010 11:25 AM, Erik Carlin wrote:
>> You are right.  The 1M number was VMs not hosts.  At least, that was
>>from
>> one scale discussion we had within Rackspace.  I'm not sure what the
>> "official" nova target limits are and I can't find anything on launchpad
>> that defines it.  If there is something, could someone please send me a
>> link.
>> 
>> I'm am certain that Google can manage more than 10K physical servers per
>> DC. Rackspace does this today.
>> 
>> If I combine what I know about EC2 and Cloud Servers, I would set the
>>ROM
>> scale targets as:
>> 
>> ABSOLUTE
>> 1M VMs
>> 50K hosts
>> 
>> RATE
>> 500K transactions/day (create/delete server, list servers, resize
>>server,
>> etc. - granted, some are more expensive than others)
>> That works out to ~21K/hr but it won't be evenly distributed.  To allow
>> for peak, I would say something like 75K/hour or ~21/sec.
>> 
>> Thoughts?
>> 
>> Erik
>> 
>> 
>> On 12/30/10 9:20 AM, "Pete Zaitcev"  wrote:
>> 
>>> On Wed, 29 Dec 2010 19:27:09 +
>>> Erik Carlin  wrote:
>>>
 The 1M host limit still seems reasonable to me. []
>>>
>>> In my opinion, such numbers are completely out of whack. Google's
>>>Chubby
>>> article says that the busiest Chubby has 90,000 clients (not hosts!)
>>>and
>>> the biggest datacenter has 10,000 systems. They found such numbers
>>> pushing the border of unmanageable. Granted they did not use
>>> virtualization, but we're talking the number of boxes in both cases.
>>>
>>> So to reach 1M hosts in a Nova instance you have to have it manage
>>> 100 datacenters. There are going to be calls for federation of Novas
>>> long before this number is reached.
>>>
>>> Sustaining a high flap rate is a worthy goal and will have an
>>> important practical impact. And having realistic sizing ideas is
>>> going to help it.
>>>
>>> -- Pete
>> 
>> 
>> 
>> Confidentiality Notice: This e-mail message (including any attached or
>> embedded documents) is intended for the exclusive and confidential use
>>of the
>> individual or entity to which this message is addressed, and unless
>>otherwise
>> expressly indicated, is confidential and privileged information of
>>Rackspace. 
>> Any dissemination, distribution or copying of the enclosed material is
>>prohibited.
>> If you receive this transmission in error, please notify us immediately
>>by e-mail
>> at ab...@rackspace.com, and delete the original message.
>> Your cooperation is appreciated.
>> 
>> 
>> ___
>> Mailing list: https://launchpad.net/~openstack
>> Post to : openstack@lists.launchpad.net
>> Unsubscribe : https://launchpad.net/~openstack
>> More help   : https://help.launchpad.net/ListHelp
>
>



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@racksp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Erik Carlin]

Jay -

Few comments below...

Erik

On 12/30/10 10:43 AM, "Jay Pipes"  wrote:

>On Wed, Dec 29, 2010 at 2:27 PM, Erik Carlin 
>wrote:
>> We know Amazon is highly, highly elastic.  While the instances launched
>> per day is impressive, we know that many of those instances have a short
>> life.
>
>OK, good point.  But, this begs the question: what should Nova's
>priority be?  Elasticity -- in other words, being able to quickly spin
>up and down hundreds of thousands of instances per day?  Or
>manageability at large scale -- in other words, a system that is easy
>to administer at hundreds of thousands of physical nodes?  Or pure
>scalability on the user end -- meaning, given a specific installation
>of applications on a given type of instance (say, m1.large), what is
>the pattern of throughput for that set of applications as the size of
>the grid increases to hundreds of thousands of physical nodes?
>
>Or do we take an ambivalent position on the above and go for some sort
>of "general scalability"?

I think we should quantify target absolute and rate limits and seek to
meet those.  I proposed a set in the email I just sent that combines what
we know about EC2 and CS today.  Would love to know what others think.

>
>> I see Guy is now teaming up with CloudKick on this report.  The EC2
>> instance ID enables precise measurement of instances launched, and
>> CloudKick provides some quantitative measure of lifetime of instances.
>> Last time I checked, those numbers we're something like 3% of EC2
>> instances launched via CK were still running (as a point of reference,
>> something like 80% of Rackspace cloud servers were still running).
>
>I see this as tangential at best, and mostly a localized issue with
>Rackspace Cloud Servers, and not something that is inherently
>important to Nova.  Let me explain.  IMHO, there are two big reasons
>why there is less "churn rate" of instances on Cloud Servers than EC2:
>
>1) Different level/type of customers
>
>RS Cloud Servers tends to attract a more "corporate" or "enterprisey"
>type of customer.  These customers tend to deploy applications into
>the RS Cloud with more permanent patterns.  Applications like
>departmental or financial applications don't tend to "disappear" or be
>experimental.
>
>2) Application bursting/overflow capacity
>
>Perhaps more important than the type of customer RS Cloud Servers
>attracts, I think many people believe that automating capacity
>bursting into the RS Cloud is more difficult than EC2 (possibly due to
>a larger feature set in the EC2 API for managing groups of servers/IP
>ranges?), and that may contribute to the lower instance churn rate.
>Due to EC2's elasticity (and "hackability" as termie would call
>it...), companies are better able to programmatically spin up
>instances to offload peak traffic from web applications and then spin
>those instances down after traffic subsides.  Perhaps if RS Cloud
>Servers had better hackability, I think you'd see the RS Cloud churn
>rate increase dramatically.
>
>These are just my thoughts, though.  I'd be interested to hear what
>other's opinions on this are.

The lifetime estimates are relevant in translating the EC2 rate numbers
(which we can quantifiably measure) to absolute host estimates.  I agree
the CS info is somewhat superfluous - I provided it more as an FYI.  The
primary reason for the instance lifetime differentiation between EC2 and
CS IMHO is the persistent vs. ephemeral nature of VMs.  Coupled with spot
instances, cluster compute instances, cluster GPU instances, etc. EC2
becomes a very nice platform for map/reduce, monte carlo simulations,
video encoding, protein folding, etc. which are transitory workloads.  I
actually think there is an equal amount of "enterprisey" customers on
both.  I agree we can and should increase the "hackability" of the CS/OS
API.  IMO, the goal of nova should be combining strong elasticity and
hackability properties with persistence, then you can run either
transitory or persistent workloads well.  If we can achieve that, nova
will meet the needs of a large and diverse set of target clouds.
 

>
>> To meet the elasticity demands of EC2, nova would need to support a high
>> change rate of adds/deletes (not to mention state polling, resizes,
>>etc).
>> Is there a nova change rate target as well or just a physical host
>>limit?
>> The 1M host limit still seems reasonable to me.  Large scale deployments
>> will break into regions where each region is an independent nova
>> deployment that each has a 1M host limit.
>
>This change rate is something that should be tracked in the continuous
>integration project.
>
>Cheers!
>-jay



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution o

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Rick Clark]

Actually it was 1M hosts and  Ivthink 45 million vms.  It was meant to
be across all regions.  Jason Seats set the number arbitrarily, but it
is a good target to not let us forget about scaling while we design.

I think eventually all loads will be more ephemeral.  So, I think I
agree with your numbers, if you are talking about a single availability
zone.

On 12/30/2010 11:25 AM, Erik Carlin wrote:
> You are right.  The 1M number was VMs not hosts.  At least, that was from
> one scale discussion we had within Rackspace.  I'm not sure what the
> "official" nova target limits are and I can't find anything on launchpad
> that defines it.  If there is something, could someone please send me a
> link.
> 
> I'm am certain that Google can manage more than 10K physical servers per
> DC. Rackspace does this today.
> 
> If I combine what I know about EC2 and Cloud Servers, I would set the ROM
> scale targets as:
> 
> ABSOLUTE
> 1M VMs
> 50K hosts
> 
> RATE
> 500K transactions/day (create/delete server, list servers, resize server,
> etc. - granted, some are more expensive than others)
> That works out to ~21K/hr but it won't be evenly distributed.  To allow
> for peak, I would say something like 75K/hour or ~21/sec.
> 
> Thoughts?
> 
> Erik
> 
> 
> On 12/30/10 9:20 AM, "Pete Zaitcev"  wrote:
> 
>> On Wed, 29 Dec 2010 19:27:09 +
>> Erik Carlin  wrote:
>>
>>> The 1M host limit still seems reasonable to me. []
>>
>> In my opinion, such numbers are completely out of whack. Google's Chubby
>> article says that the busiest Chubby has 90,000 clients (not hosts!) and
>> the biggest datacenter has 10,000 systems. They found such numbers
>> pushing the border of unmanageable. Granted they did not use
>> virtualization, but we're talking the number of boxes in both cases.
>>
>> So to reach 1M hosts in a Nova instance you have to have it manage
>> 100 datacenters. There are going to be calls for federation of Novas
>> long before this number is reached.
>>
>> Sustaining a high flap rate is a worthy goal and will have an
>> important practical impact. And having realistic sizing ideas is
>> going to help it.
>>
>> -- Pete
> 
> 
> 
> Confidentiality Notice: This e-mail message (including any attached or
> embedded documents) is intended for the exclusive and confidential use of the
> individual or entity to which this message is addressed, and unless otherwise
> expressly indicated, is confidential and privileged information of Rackspace. 
> Any dissemination, distribution or copying of the enclosed material is 
> prohibited.
> If you receive this transmission in error, please notify us immediately by 
> e-mail
> at ab...@rackspace.com, and delete the original message. 
> Your cooperation is appreciated.
> 
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp




signature.asc
Description: OpenPGP digital signature
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Erik Carlin]

You are right.  The 1M number was VMs not hosts.  At least, that was from
one scale discussion we had within Rackspace.  I'm not sure what the
"official" nova target limits are and I can't find anything on launchpad
that defines it.  If there is something, could someone please send me a
link.

I'm am certain that Google can manage more than 10K physical servers per
DC. Rackspace does this today.

If I combine what I know about EC2 and Cloud Servers, I would set the ROM
scale targets as:

ABSOLUTE
1M VMs
50K hosts

RATE
500K transactions/day (create/delete server, list servers, resize server,
etc. - granted, some are more expensive than others)
That works out to ~21K/hr but it won't be evenly distributed.  To allow
for peak, I would say something like 75K/hour or ~21/sec.

Thoughts?

Erik


On 12/30/10 9:20 AM, "Pete Zaitcev"  wrote:

>On Wed, 29 Dec 2010 19:27:09 +
>Erik Carlin  wrote:
>
>> The 1M host limit still seems reasonable to me. []
>
>In my opinion, such numbers are completely out of whack. Google's Chubby
>article says that the busiest Chubby has 90,000 clients (not hosts!) and
>the biggest datacenter has 10,000 systems. They found such numbers
>pushing the border of unmanageable. Granted they did not use
>virtualization, but we're talking the number of boxes in both cases.
>
>So to reach 1M hosts in a Nova instance you have to have it manage
>100 datacenters. There are going to be calls for federation of Novas
>long before this number is reached.
>
>Sustaining a high flap rate is a worthy goal and will have an
>important practical impact. And having realistic sizing ideas is
>going to help it.
>
>-- Pete



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Jay Pipes]

On Wed, Dec 29, 2010 at 2:27 PM, Erik Carlin  wrote:
> We know Amazon is highly, highly elastic.  While the instances launched
> per day is impressive, we know that many of those instances have a short
> life.

OK, good point.  But, this begs the question: what should Nova's
priority be?  Elasticity -- in other words, being able to quickly spin
up and down hundreds of thousands of instances per day?  Or
manageability at large scale -- in other words, a system that is easy
to administer at hundreds of thousands of physical nodes?  Or pure
scalability on the user end -- meaning, given a specific installation
of applications on a given type of instance (say, m1.large), what is
the pattern of throughput for that set of applications as the size of
the grid increases to hundreds of thousands of physical nodes?

Or do we take an ambivalent position on the above and go for some sort
of "general scalability"?

> I see Guy is now teaming up with CloudKick on this report.  The EC2
> instance ID enables precise measurement of instances launched, and
> CloudKick provides some quantitative measure of lifetime of instances.
> Last time I checked, those numbers we're something like 3% of EC2
> instances launched via CK were still running (as a point of reference,
> something like 80% of Rackspace cloud servers were still running).

I see this as tangential at best, and mostly a localized issue with
Rackspace Cloud Servers, and not something that is inherently
important to Nova.  Let me explain.  IMHO, there are two big reasons
why there is less "churn rate" of instances on Cloud Servers than EC2:

1) Different level/type of customers

RS Cloud Servers tends to attract a more "corporate" or "enterprisey"
type of customer.  These customers tend to deploy applications into
the RS Cloud with more permanent patterns.  Applications like
departmental or financial applications don't tend to "disappear" or be
experimental.

2) Application bursting/overflow capacity

Perhaps more important than the type of customer RS Cloud Servers
attracts, I think many people believe that automating capacity
bursting into the RS Cloud is more difficult than EC2 (possibly due to
a larger feature set in the EC2 API for managing groups of servers/IP
ranges?), and that may contribute to the lower instance churn rate.
Due to EC2's elasticity (and "hackability" as termie would call
it...), companies are better able to programmatically spin up
instances to offload peak traffic from web applications and then spin
those instances down after traffic subsides.  Perhaps if RS Cloud
Servers had better hackability, I think you'd see the RS Cloud churn
rate increase dramatically.

These are just my thoughts, though.  I'd be interested to hear what
other's opinions on this are.

> To meet the elasticity demands of EC2, nova would need to support a high
> change rate of adds/deletes (not to mention state polling, resizes, etc).
> Is there a nova change rate target as well or just a physical host limit?
> The 1M host limit still seems reasonable to me.  Large scale deployments
> will break into regions where each region is an independent nova
> deployment that each has a 1M host limit.

This change rate is something that should be tracked in the continuous
integration project.

Cheers!
-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Bret Piatt]

We can break this down into a list of measureable test cases.  Many of these
tests aren't just single host system tests.  They require an integrated
deployment to find and eliminate the bottleneck.  I'm certain I'm missing
additional items we would want to measure so consider this an off the top of
my head incomplete list.

Rate of change tests:
1. Maximum rate of change per host machine -- this could be create, delete,
migrate, snapshot, backup.
2. Maximum number of host machines per host controller machine that it can
sustain at their maximum rate of change.
3. Maximum number of host machines per glance machine that it can sustain at
their maximum rate of change.
4. Maximum number of requests per second and total buffer of requests on the
message queue per machine.
5. Maximum number of storage volume operations per storage controller
machine.

Scale tests:
1. Maximum number of VMs on a host machine.
2. Maximum number of VMs per host controller.
3. Maximum number of storage volumes attached to a host machine.
4. Maximum number of storage volumes per storage controller machine.
5. Maximum number of VM records in the cloud database.
6. Maximum number of storage volume records in the cloud database.
7. Maximum number of VM images managed per glance machine.

I understand that these maximum numbers will vary greatly depending on what
"a machine", the size of the test VM image, the network configuration, and
many other factors.  I propose we get a test bed setup to measure our own
CloudMIPS (call it whatever we want) like a BogoMIPS so we can measure the
performance of the codebase over time.  We can run the tests each weekend on
the trunk as of 00:00 GMT Saturday and by tracking it against the code
merged each week ensure we're headed in the right direction and if not we'll
be consciously aware of which changes impacted the system performance.  I'm
up for organizing the effort -- finding hardware, DC space, and getting
operational support to manage it.  Is anyone interested in participating?

Back on the AWS example I'll make some sweeping generalizations and round
numbers off to make the math easy.  I'll assume 50k hosts in the measured
facility with a peak rate of change of 150,000/day.  That is only a required
rate of change per host machine of 1 VM / 8 hours.  Even if they only have
12.5k hosts in that facility that is 1 VM / 2 hours per machine.  If they
have 20 VMs sustained then we're looking at 250k to 1M VMs for scale.  

Bret Piatt
OpenStack.org
Twitter: @bpiatt / Mobile 210-867-9338
Open Source Cloud Infrastructure: http://www.openstack.org



-Original Message-
From: openstack-bounces+bret=openstack@lists.launchpad.net
[mailto:openstack-bounces+bret=openstack@lists.launchpad.net] On Behalf
Of Ewan Mellor
Sent: Thursday, December 30, 2010 7:48 AM
To: Jay Pipes; openstack@lists.launchpad.net
Subject: Re: [Openstack] Some insight into the number of instances Nova
needs to spin up...

Those are interesting figures, but it would also be interesting to know how
many of those are kept running.  Our 1M host target is irrelevant if all
those instances are shut down again a day later.  If Amazon are adding *and
keeping* 70k VMs a day then that's a very different matter.

Ewan.

> -Original Message-
> From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net
> [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net]
> On Behalf Of Jay Pipes
> Sent: 29 December 2010 16:47
> To: openstack@lists.launchpad.net
> Subject: [Openstack] Some insight into the number of instances Nova 
> needs to spin up...
> 
> Some insight into the number of instances being spun up *per day* on 
> AWS, *just in the US-East-1 region*:
> 
> http://www.jackofallclouds.com/2010/12/recounting-ec2/
> 
> Avg in 2010: 70,528 instances spun up each day
> Max: 150,800 instances in a single day
> 
> Food for thought.
> 
> What do we think Nova could/can support? I know we are aiming at 
> supporting clouds of 1M physical hosts. Perhaps we need to re-assess?
> :)
> 
> /me urges more prioritization of the openstack-ci (continuous 
> integration and performance/stress testing) project in Cactus...
> 
> -jay
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Pete Zaitcev]

On Wed, 29 Dec 2010 19:27:09 +
Erik Carlin  wrote:

> The 1M host limit still seems reasonable to me. []

In my opinion, such numbers are completely out of whack. Google's Chubby
article says that the busiest Chubby has 90,000 clients (not hosts!) and
the biggest datacenter has 10,000 systems. They found such numbers
pushing the border of unmanageable. Granted they did not use
virtualization, but we're talking the number of boxes in both cases.

So to reach 1M hosts in a Nova instance you have to have it manage
100 datacenters. There are going to be calls for federation of Novas
long before this number is reached.

Sustaining a high flap rate is a worthy goal and will have an
important practical impact. And having realistic sizing ideas is
going to help it.

-- Pete

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

[Openstack] Housekeeping specs

[Thierry Carrez]

Hi everyone,

We have a few housekeeping tasks that are basically a shared effort and
never completely ended, like improving our test code coverage,
docstrings or pylint scores.

Generic blueprints are very bad at tracking that kind of effort, since
those are not assigned to specific people, or targeted against a
specific release. However, not having any tracking generally result in
those tasks being ignored, while they make really nice goals for
newcomers in the project.

So I propose to keep them as blueprints, not targeting them to a
specific release, and assign them to the project teams (~nova, ~swift,
etc). That way we can find them by looking at specs assigned to generic
teams, like for nova:

https://blueprints.launchpad.net/~nova/+specs?role=assignee

I exposed that development task in the
http://wiki.openstack.org/HowToContribute doc so that newcomers can find
them.

If you have any housekeeping task that should be on that list, feel free
to ping me, or directly file a blueprint / assign the project team to
them. Should this one be part of the list, for example ?

https://blueprints.launchpad.net/nova/+spec/unittest-coverage

We should review the progress on those at each design summit to see if
that's successful enough, or if we need to change how we track them.

Cheers, and happy new year ;)

-- 
Thierry Carrez (ttx)
Release Manager, OpenStack

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Some insight into the number of instances Nova needs to spin up...

[Ewan Mellor]

Those are interesting figures, but it would also be interesting to know how 
many of those are kept running.  Our 1M host target is irrelevant if all those 
instances are shut down again a day later.  If Amazon are adding *and keeping* 
70k VMs a day then that's a very different matter.

Ewan.

> -Original Message-
> From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net
> [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net]
> On Behalf Of Jay Pipes
> Sent: 29 December 2010 16:47
> To: openstack@lists.launchpad.net
> Subject: [Openstack] Some insight into the number of instances Nova
> needs to spin up...
> 
> Some insight into the number of instances being spun up *per day* on
> AWS, *just in the US-East-1 region*:
> 
> http://www.jackofallclouds.com/2010/12/recounting-ec2/
> 
> Avg in 2010: 70,528 instances spun up each day
> Max: 150,800 instances in a single day
> 
> Food for thought.
> 
> What do we think Nova could/can support? I know we are aiming at
> supporting clouds of 1M physical hosts. Perhaps we need to re-assess?
> :)
> 
> /me urges more prioritization of the openstack-ci (continuous
> integration and performance/stress testing) project in Cactus...
> 
> -jay
> 
> ___
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

Re: [Openstack] Easy API

[Jorge Williams]

Hi Andy,

>From your description, I don't think we're that far apart at all.  In fact, I 
>think we're proposing almost exactly the same thing -- at least in terms of 
>auth.   The purpose of our blueprint is *not* to provide a framework for 
>creating an authentication service, but rather to simply establish a protocol 
>between an authentication system and the underlying API.  You've done the same 
>thing.  You suggest using X-OpenStack-User to pass the user to the API, we 
>suggest using X-Authorization -- end of the day, it's the same approach.

Sure, in our spec, we spend quite a bit talking about error conditions and 
special cases because we want to ensure consistency and we want to make sure 
that the same authentication frameworks can be used between different APIs, 
etc.  We also want to make sure we have the flexibility to support multiple 
authentication schemes.

As you suggest, though, the typical case is really simple:  you have a 
middleware component that understands some authentication protocol and passes 
the information to the rest of the system.  You can implement our spec in only 
a few lines a code.  Why not just standardize on the headers?  If you do so you 
can simply rely on our blueprint to solve the authentication part for you.

-jOrGe W.

On Dec 30, 2010, at 4:05 AM, Andy Smith wrote:

Hi Jorge,

I don't really think we are on the same page for very long, your spec goes 
quite far down the path of specific authorization schemes where as mine really 
only takes the first step.

In general the spirit seems to be the same, that we want another service to be 
able to handle authentication, but our approaches are rather different, where 
yours provides a framework for creating an authentication service, mine is 
simply stating the information it needs and requesting that somebody else 
provide it.

Whether the authentication framework is OpenID, some form of PKI, or Facebook 
Auth the only important part to my spec is that at some point it boils down to 
the information the rest of the system actually needs to determine permissions, 
and at current those are two things: user and project.

While the processes you've described are some of the varied ways to implement 
something that could sit in front of those two basic necessities I think the 
scope of the specs are quite different.

I personally don't think the problem is so complex as your spec describes in 
the majority of the cases (most people will simply put a standard 
authenticating proxy / middleware backended on their user / project datastore 
in front of the api) but I do think your spec could provide a specific 
implementation pattern of an authentication component as fits the needs of, for 
example, rackspace or some other provider with a similar scheme that sits in 
front of the API.

--andy

On Thu, Dec 30, 2010 at 12:37 AM, Jorge Williams 
mailto:jorge.willi...@rackspace.com>> wrote:
Hi Andy,

The delegated auth pattern in Easy API seems to match our existing blueprint 
for authentication in OpenStack.

blueprint: https://blueprints.launchpad.net/nova/+spec/nova-authn
spec:  http://wiki.openstack.org/openstack-authn

Have you taken a look at the blueprint?  Are we on the same page in terms of 
auth or are you proposing something different?

-jOrGe W.

On Dec 29, 2010, at 2:08 PM, Andy Smith wrote:

+openstack

On Wed, Dec 29, 2010 at 12:06 PM, Andy Smith 
mailto:andys...@gmail.com>> wrote:
Heya Matt,

I was intending on writing an email to the mailing list about it (so I'll just 
CC it), was just going over tone with the Anso team first since I tend to come 
off a bit antagonistic.

The short summary is that we wanted to return "hackability" to the project, as 
developers the move towards API compatibility has slowly removed most of the 
tools we had in place to test out new features as they were being worked on, so 
in that way the reflection api, self-registration of services, command-line 
tool and general simplicity (you can explain how everything works in just a few 
sentences) are a solution.

Part of that hackability also meant hackability for third-parties, we wanted 
developers to be able to add functionality to Nova without having to go through 
the Nova review process and even be able package that functionality separately, 
this is an obvious  goal for any large entity using openstack internally where 
they have additional changes to make to customize their deployment.

Easy API makes up a good basis for extensibility of the project over time.

The existing APIs are still necessary to serve their purpose which is 
translating the intents defined by existing specifications/implementations to 
tasks that make sense for Nova, and should it be desirable those interfaces can 
easily be built on top of and decoupled (due to the delegated auth pattern) 
from the Easy API, the EasyCloudTestCase gives a decent example of how such a 
thing could be done.

In my personal opinion I see the two existing API implementati

Re: [Openstack] Easy API

[Andy Smith]

Hi Jorge,

I don't really think we are on the same page for very long, your spec goes
quite far down the path of specific authorization schemes where as mine
really only takes the first step.

In general the spirit seems to be the same, that we want another service to
be able to handle authentication, but our approaches are rather different,
where yours provides a framework for creating an authentication service,
mine is simply stating the information it needs and requesting that somebody
else provide it.

Whether the authentication framework is OpenID, some form of PKI, or
Facebook Auth the only important part to my spec is that at some point it
boils down to the information the rest of the system actually needs to
determine permissions, and at current those are two things: user and
project.

While the processes you've described are some of the varied ways to
implement something that could sit in front of those two basic necessities I
think the scope of the specs are quite different.

I personally don't think the problem is so complex as your spec describes in
the majority of the cases (most people will simply put a standard
authenticating proxy / middleware backended on their user / project
datastore in front of the api) but I do think your spec could provide a
specific implementation pattern of an authentication component as fits the
needs of, for example, rackspace or some other provider with a similar
scheme that sits in front of the API.

--andy

On Thu, Dec 30, 2010 at 12:37 AM, Jorge Williams <
jorge.willi...@rackspace.com> wrote:

>  Hi Andy,
>
>  The delegated auth pattern in Easy API seems to match our existing
> blueprint for authentication in OpenStack.
>
>  blueprint: https://blueprints.launchpad.net/nova/+spec/nova-authn
> spec:  http://wiki.openstack.org/openstack-authn
>
>  Have you taken a look at the blueprint?  Are we on the same page in terms
> of auth or are you proposing something different?
>
>  -jOrGe W.
>
>  On Dec 29, 2010, at 2:08 PM, Andy Smith wrote:
>
> +openstack
>
> On Wed, Dec 29, 2010 at 12:06 PM, Andy Smith  wrote:
>
>> Heya Matt,
>>
>>  I was intending on writing an email to the mailing list about it (so
>> I'll just CC it), was just going over tone with the Anso team first since I
>> tend to come off a bit antagonistic.
>>
>>  The short summary is that we wanted to return "hackability" to the
>> project, as developers the move towards API compatibility has slowly removed
>> most of the tools we had in place to test out new features as they were
>> being worked on, so in that way the reflection api, self-registration of
>> services, command-line tool and general simplicity (you can explain how
>> everything works in just a few sentences) are a solution.
>>
>>  Part of that hackability also meant hackability for third-parties, we
>> wanted developers to be able to add functionality to Nova without having to
>> go through the Nova review process and even be able package that
>> functionality separately, this is an obvious  goal for any large entity
>> using openstack internally where they have additional changes to make to
>> customize their deployment.
>>
>>  Easy API makes up a good basis for extensibility of the project over
>> time.
>>
>>  The existing APIs are still necessary to serve their purpose which is
>> translating the intents defined by existing specifications/implementations
>> to tasks that make sense for Nova, and should it be desirable those
>> interfaces can easily be built on top of and decoupled (due to the delegated
>> auth pattern) from the Easy API, the EasyCloudTestCase gives a decent
>> example of how such a thing could be done.
>>
>>  In my personal opinion I see the two existing API implementations as
>> 'compatibility' APIs where as Easy API is a direct API and much easier to
>> understand because of it.
>>
>>  The relevant blueprint (read the spec, obvs):
>> https://blueprints.launchpad.net/nova/+spec/easy-api
>>
>>  --andy
>>
>> On Wed, Dec 29, 2010 at 11:20 AM, Matt Dietz wrote:
>>
>>>  Hey guys,
>>>
>>>  I was wondering if you could answer a few questions? I get the argument
>>> that EC2 and eucatools suck, and that the OS API is incomplete/unstable.
>>> What prompted you starting down the Easy API path? Subsequently, what issues
>>> are you hoping to solve with it? From what I can see, it's the
>>> WADL-like/reflection interface that seems to be the driving force. Is this
>>> correct?
>>> I'm curious mostly because a lot of effort has been expended with the
>>> goal of making the existing OS API the canonical one. I'm fine with a better
>>> effort, but I've been operating in accordance with the internal goal of
>>> making it work 100% like the Rackspace API so all existing bindings will
>>> "just work" if someone cares to try out Openstack. Obviously we can't
>>> continue working on parallel paths because we'll just end up fracturing the
>>>  API user-base until we can decide which API is going to stick.
>>> I'd like to get the discussion 

Re: [Openstack] Easy API

[Jorge Williams]

Hi Andy,

The delegated auth pattern in Easy API seems to match our existing blueprint 
for authentication in OpenStack.

blueprint: https://blueprints.launchpad.net/nova/+spec/nova-authn
spec:  http://wiki.openstack.org/openstack-authn

Have you taken a look at the blueprint?  Are we on the same page in terms of 
auth or are you proposing something different?

-jOrGe W.

On Dec 29, 2010, at 2:08 PM, Andy Smith wrote:

+openstack

On Wed, Dec 29, 2010 at 12:06 PM, Andy Smith 
mailto:andys...@gmail.com>> wrote:
Heya Matt,

I was intending on writing an email to the mailing list about it (so I'll just 
CC it), was just going over tone with the Anso team first since I tend to come 
off a bit antagonistic.

The short summary is that we wanted to return "hackability" to the project, as 
developers the move towards API compatibility has slowly removed most of the 
tools we had in place to test out new features as they were being worked on, so 
in that way the reflection api, self-registration of services, command-line 
tool and general simplicity (you can explain how everything works in just a few 
sentences) are a solution.

Part of that hackability also meant hackability for third-parties, we wanted 
developers to be able to add functionality to Nova without having to go through 
the Nova review process and even be able package that functionality separately, 
this is an obvious  goal for any large entity using openstack internally where 
they have additional changes to make to customize their deployment.

Easy API makes up a good basis for extensibility of the project over time.

The existing APIs are still necessary to serve their purpose which is 
translating the intents defined by existing specifications/implementations to 
tasks that make sense for Nova, and should it be desirable those interfaces can 
easily be built on top of and decoupled (due to the delegated auth pattern) 
from the Easy API, the EasyCloudTestCase gives a decent example of how such a 
thing could be done.

In my personal opinion I see the two existing API implementations as 
'compatibility' APIs where as Easy API is a direct API and much easier to 
understand because of it.

The relevant blueprint (read the spec, obvs): 
https://blueprints.launchpad.net/nova/+spec/easy-api

--andy

On Wed, Dec 29, 2010 at 11:20 AM, Matt Dietz 
mailto:matt.di...@rackspace.com>> wrote:
Hey guys,

I was wondering if you could answer a few questions? I get the argument that 
EC2 and eucatools suck, and that the OS API is incomplete/unstable. What 
prompted you starting down the Easy API path? Subsequently, what issues are you 
hoping to solve with it? From what I can see, it's the WADL-like/reflection 
interface that seems to be the driving force. Is this correct?
I'm curious mostly because a lot of effort has been expended with the goal of 
making the existing OS API the canonical one. I'm fine with a better effort, 
but I've been operating in accordance with the internal goal of making it work 
100% like the Rackspace API so all existing bindings will "just work" if 
someone cares to try out Openstack. Obviously we can't continue working on 
parallel paths because we'll just end up fracturing the  API user-base until we 
can decide which API is going to stick.
I'd like to get the discussion started, and I'll be glad to take it to the 
blueprint (since I see you made one as of 18 hours ago) or IRC.

Thanks,

Matt


___
Mailing list: https://launchpad.net/~openstack
Post to : 
openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp