Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
Licensing - Has there been a conversation around licensing? Glance may not be the place to manage licenses (issue, track, revoke, renew, register, etc...) but one attribute of an image is whether it needs a license or not and if it does, where does one go to get one. Licensing service? Maybe it can start in Glance... Z On Apr 12, 2011, at 1:11 PM, Jay Pipes wrote: Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: * Authn/authz - This is a big one, but dependent on the overall discussion of federated auth going on in the Nova/Swift communities. Glance will merely follow suit with what Nova does most likely. * Image conversion. This actually already has a blueprint, but maybe good for a detailed discussion at the summit? See https://blueprints.launchpad.net/glance/+spec/image-file-conversion * Metrics - for instance, tracking operations performed (read/write, bytes out/in, ?) Would this even be useful? * Integration with more backend storage systems? * XML support in the API? * Having Glance understand what is contained in the disk images by inspecting them on upload? * A Glance dashboard app? Please feel free to expand on any of the above and add any suggestions you have on the future direction of Glance. Your input is truly appreciated. Cheers! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
Hi, let me say something since this topic touches the feature that we contribute. -Original Message- From: openstack-bounces+masumotok=nttdata.co...@lists.launchpad.net [mailto:openstack-bounces+masumotok=nttdata.co...@lists.launchpad.net] On Behalf Of Brian Schott Sent: Thursday, April 14, 2011 7:14 AM To: Jay Pipes Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] distributed and heterogeneous schedulers Jay, :( I've brought this up before as well. The term metadata is used incorrectly to refer to custom key/value attributes of something instead of referring to data about the data (for instance, the type and length constraints of a data field). We could move the cpu_info, xpu_info, and net_info fields off to respective -metadata tables, but that would involve lots of changes to NTT Data's live migration additions with respect to cpu_info. Not sure the USC-ISI team would want to break/own that... I don't think there is much more about a compute node that needs to be added to Instance Types, Compute Nodes, and Instances than cpus, memory, disk, net, accelerators. Unfortunately, because the OpenStack API uses the actual term metadata in the API, that's what the table was named and that's how key/value pairs are referred to in the code. We have at least three choices here: 1) Continue to add fields to the instances table (or compute_nodes table) for these main attributes like cpu_arch, etc. 2) Use the custom key/value table (instance_metadata) to store these attribute names and their values 3) Do both 1) and 2) I would prefer that we use 1) above for fields that are common to all nodes (and thus can be NOT NULL fields in the database and be properly indexed. And all other attributes that are not common to all nodes use the instance_metadata table. Thoughts? Agree with 1). Accelerators is the only thing that is somewhat optional, so that column might be sparse. I've been looking for how instance_metadata is being used. Our test deployment has a blank table, but not sure it is currently running. Any recommendation on how to represent the supported values? All I found googling was: https://blueprints.launchpad.net/nova/+spec/extra-data/ My understanding is that instance metadata table is necessary and should be (k,v) type since many people wants to add various info to instance table, then a column is NULL except for a specific purpose, therefore we need (k,v) type of declaration. On the other hand, compute nodes table doesn’t have so many column so far. So regarding to compute node table, I also agree with 1). When it comes to adding many columns to compute nodes table(including someone has a plan to do it), it may be better to discuss. Then NTTData team willingly contributes to change live migration code followed by upcoming discussion :) - Will your capabilities scheduler, constraint scheduler, and/or distributed schedulers understand different available hardware resources on compute nodes? I was assuming they would understand different available hardware resources by querying a database table that housed attributes pertaining to a single host or a group of hosts (a zone). I see, the goal is to be able to build dynamic REST requests for features based on queries into the metadata service. As long as we can inject a -t cg1.4xlarge at one end and have that get routed to a compute node with GPU hardware on the other end, we're not tied to the centralized database implementation. I don't see how having the database implementation be centralized or not affects the above statement. Could you elaborate? Sure, I meant that if the distributed scheduler devolves entirely into posting schedule requests on rabbit-mq and letting each individual compute node be its own zone scheduler, that we still need these shorthand instance type mechanisms to advertise certain configurations both for compatibility with ec2 API and as a marketing tool. Walmart offers 3 kinds of toasters because the cheap one gets you in the door, the expensive one has the most gee-wiz appeal, and the medium one has a profit margin designed to maximize revenue. In this analogy, nodes are toasters :-). PS: I sent this to the mailing list a week ago and didn't get a reply, now can't even find this in the openstack list archive. Anyone else having their posts quietly rejected? I saw the original, if you are referring to this one: https://lists.launchpad.net/openstack/msg01645.html Nope, that was a different email. Not a big deal. I have 3 accounts and can't keep them straight. Brian ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list:
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
How about QA? Is this something that you'd want to consider specifically for Glance? It's the newest of our services, and the smallest team, so I think that it's worthwhile explicitly blocking out some time to look at automated testing in an integrated environment. Given the size of the images involved, it won't take much to put some serious stress on Glance, so some scalability / stress testing would be really valuable. Cheers, Ewan. -Original Message- From: openstack-bounces+ewan.mellor=citrix@lists.launchpad.net [mailto:openstack-bounces+ewan.mellor=citrix@lists.launchpad.net] On Behalf Of Jay Pipes Sent: 12 April 2011 19:12 To: openstack@lists.launchpad.net Subject: [Openstack] Enhancements to Glance in Diablo? Input welcomed Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: * Authn/authz - This is a big one, but dependent on the overall discussion of federated auth going on in the Nova/Swift communities. Glance will merely follow suit with what Nova does most likely. * Image conversion. This actually already has a blueprint, but maybe good for a detailed discussion at the summit? See https://blueprints.launchpad.net/glance/+spec/image-file-conversion * Metrics - for instance, tracking operations performed (read/write, bytes out/in, ?) Would this even be useful? * Integration with more backend storage systems? * XML support in the API? * Having Glance understand what is contained in the disk images by inspecting them on upload? * A Glance dashboard app? Please feel free to expand on any of the above and add any suggestions you have on the future direction of Glance. Your input is truly appreciated. Cheers! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
Let's not confuse instance metadata with Compute Node Capabilities. When scheduling, the instance has not been created yet. We have to make decisions on where the instance will ultimately reside on a number of factors: 1. the capabilities of the host hypervisor (the Compute node) 2. the current load the host is under 3. the distribution policies of the customer (ie. don't put all my instances on the same host/zone, etc) Capabilities are just multi-value key-value pairs, such as: can_host=linux;windows, cpu_type=gpu, magic_sauce=purple,blue,red; cpu_min_max=0.01,0.98; Zones today have two sets of capabilities: 1. Zone capabilities 2. Host (Compute) capabilities These are largely static and defined when a new zone or Compute node is stood up. There are also dynamic capabilities, such as cpu load, available disk, used network, etc. These are reported from the Services (compute, volume, network, etc) periodically and collected by the Scheduler services in a thing called the Zone Manager. The Scheduler drivers can use this data in its strategies. The ZM also collects the capabilities (static and dynamic) from the child zones associated with this zone. From all of this information, it should be pretty easy to create a scheduler driver to suit your needs. The Distributed Scheduler blueprint, which Ed and I are working on currently, is a very generic, but flexible scheduling driver modeled on the Zone Best Match and Server Best Match algorithms Rackspace currently uses. Hope it helps! -Sandy Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cactus RCFreeze now in effect
Thanks, Thierry, for all your assistance during the last couple weeks organizing the various freezes. It's a thankless job and often very frustrating. You keep your cool at all times. Thanks for that! -jay On Thu, Apr 14, 2011 at 4:46 AM, Thierry Carrez thie...@openstack.org wrote: Hello everyone, Cactus RCFreeze is upon us, though due to Murphy's law the RC milestone will be delayed until Launchpad fully recovers from a major outage. A last Nova branch should be merged before cutting that milestone: https://code.launchpad.net/~jaypipes/nova/bug760289 Then (unless we discover a major regression in the tarball) the RC milestone will be promoted tomorrow as the Cactus release, and the Diablo floodgates will be open for unlimited merging fun ! More details at: http://wiki.openstack.org/RCFreeze Thanks for your help, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cactus RCFreeze now in effect
Hear Hear! From: Jay Pipes [jaypi...@gmail.com] Thanks, Thierry, for all your assistance during the last couple weeks organizing the various freezes. It's a thankless job and often very frustrating. You keep your cool at all times. Thanks for that! -jay Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Cactus RC milestone released
Hello everyone, A release candidate milestone for Cactus was just released. You can find it at: https://launchpad.net/nova/cactus/cactus-rc https://launchpad.net/glance/cactus/cactus-rc https://launchpad.net/swift/1.3/1.3-rc Please test and report any bug, so that we can document the known issues correctly at: http://wiki.openstack.org/ReleaseNotes/Cactus Unless we find a last-minute regression, the RC code should be shipped tomorrow as the 2011.2 / 1.3.0 final release. Cheers, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
I posted a Blueprint yesterday with a requirement for some general-purpose filters for listing images: https://blueprints.launchpad.net/glance/+spec/query-filters These are needed at Rackspace to meet feature parity with our current systems. We need to be able to create various subsets of image collections, and IMHO it makes more sense to do this on the server side than at the client. For example, we want a customer to get a list of their images for restoring from a backup. For our managed customers, we need to select only base images that have a managed cloud flag set in the metadata. We need to be able to set a minimum RAM requirement on certain images and validate that the requested VM is large enough to hold them. Some of this may already be in Glance, but I'm not that familiar with it (and trying to get up to speed). On 4/12/11 1:11 PM, Jay Pipes jaypi...@gmail.com wrote: Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: * Authn/authz - This is a big one, but dependent on the overall discussion of federated auth going on in the Nova/Swift communities. Glance will merely follow suit with what Nova does most likely. * Image conversion. This actually already has a blueprint, but maybe good for a detailed discussion at the summit? See https://blueprints.launchpad.net/glance/+spec/image-file-conversion * Metrics - for instance, tracking operations performed (read/write, bytes out/in, ?) Would this even be useful? * Integration with more backend storage systems? * XML support in the API? * Having Glance understand what is contained in the disk images by inspecting them on upload? * A Glance dashboard app? Please feel free to expand on any of the above and add any suggestions you have on the future direction of Glance. Your input is truly appreciated. Cheers! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Commas and semicolons
On 4/14/11 6:19 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: Capabilities are just multi-value key-value pairs, such as: can_host=linux;windows, cpu_type=gpu, magic_sauce=purple,blue,red; cpu_min_max=0.01,0.98; I don't like the use of commas and semicolons, and you just made the error here that I was worried about. A semicolon has a higher precedence than a comma, if you will, and is used to separate clauses. You used a semicolon between linux;windows in the above example, then commas between the other clauses, but forgot on the magic_sauce and cpu_min_max clauses. IMHO we should use commas to separate the various values, and semicolons in between the keys. Thoughts? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Commas and semicolons
Sorry, typed it quickly. There was a discussion on this on the ML previously. It's semi-colons between the values and commas between the keys foo=a;b;c, bar=d;e;f and the more-common case of single value would be foo=a, bar=b ... this keeps it compatible with DEFINE_list() of gflags. -S From: Glen Campbell Sent: Thursday, April 14, 2011 10:37 AM To: Sandy Walsh; openstack@lists.launchpad.net Subject: Commas and semicolons On 4/14/11 6:19 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: Capabilities are just multi-value key-value pairs, such as: can_host=linux;windows, cpu_type=gpu, magic_sauce=purple,blue,red; cpu_min_max=0.01,0.98; I don't like the use of commas and semicolons, and you just made the error here that I was worried about. A semicolon has a higher precedence than a comma, if you will, and is used to separate clauses. You used a semicolon between linux;windows in the above example, then commas between the other clauses, but forgot on the magic_sauce and cpu_min_max clauses. IMHO we should use commas to separate the various values, and semicolons in between the keys. Thoughts? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
1) Continue to add fields to the instances table (or compute_nodes table) for these main attributes like cpu_arch, etc. 2) Use the custom key/value table (instance_metadata) to store these attribute names and their values 3) Do both 1) and 2) I've no particular preference here, but if we start adding attributes to instance_metadata that nova code specifically knows about, we need to ensure there is a good way to tell the difference between nova metadata and user metadata. I do not believe key prefixes will provide this difference cleanly. Currently, instance_metadata is for user-provided attributes only and nova is agnostic to their contents. In the api, they are presented as such, and there are quotas on the number of metadata items the user can set. This would all break if we start adding nova metadata without further api and db (small) changes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
From: Soren Hansen [so...@openstack.org] 2. the current load the host is under I still question the usefulness of nr. 2. A host that is almost completely idle right now might be under tremendous pressure a minute from now and vice versa. Even if we had useful statistics (and trend analysis) of the existing VM's on the host, we have no data at all on the requirements of the instance we're about to launch. It seems to me that the current load only affects the time it will take to spin up the new instance, and as such says nothing about its long term (or even medium or short term!) suitability as the new instance's home. True. The last-known-load is just a suggestion for the routing, the scheduler can choose to use it or not. As with all scheduling, it will only *attempt* to create the instance on that host. But if it fails it will have list of candidates to try in weighted order. Moreso, this weighting will span zones ... not just the hosts in a single zone. That's the stuff we're working on now in the distributed-scheduler branch. We do, however, know the requirements of the instance we're about to launch based on the Flavor. This gives us bandwidth, memory and cpu limits which we can enforce in the host. Let the hypervisor handle the rest. We can also add fudge factors to the capabilities to leave ceiling space for misbehaved instances if need be (though most service providers may not like that since that costs $$) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
Yes, he and I have been emailing. I'd like to see it more generalized, rather than having only a limited set of attributes. On 4/14/11 8:45 AM, Jay Pipes jaypi...@gmail.com wrote: Hi Glen! Actually, the excellent Brian Waldon has already started working on this specifically: https://blueprints.launchpad.net/glance/+spec/api-results-filtering The blueprint was originally scheduled for Cactus, but we just didn't get around to it. Think we can combine the two blueprints? Feel free to add any input on the whiteboard for the above blueprint! Cheers! -jay On Thu, Apr 14, 2011 at 9:32 AM, Glen Campbell glen.campb...@rackspace.com wrote: I posted a Blueprint yesterday with a requirement for some general-purpose filters for listing images: https://blueprints.launchpad.net/glance/+spec/query-filters These are needed at Rackspace to meet feature parity with our current systems. We need to be able to create various subsets of image collections, and IMHO it makes more sense to do this on the server side than at the client. For example, we want a customer to get a list of their images for restoring from a backup. For our managed customers, we need to select only base images that have a managed cloud flag set in the metadata. We need to be able to set a minimum RAM requirement on certain images and validate that the requested VM is large enough to hold them. Some of this may already be in Glance, but I'm not that familiar with it (and trying to get up to speed). On 4/12/11 1:11 PM, Jay Pipes jaypi...@gmail.com wrote: Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: * Authn/authz - This is a big one, but dependent on the overall discussion of federated auth going on in the Nova/Swift communities. Glance will merely follow suit with what Nova does most likely. * Image conversion. This actually already has a blueprint, but maybe good for a detailed discussion at the summit? See https://blueprints.launchpad.net/glance/+spec/image-file-conversion * Metrics - for instance, tracking operations performed (read/write, bytes out/in, ?) Would this even be useful? * Integration with more backend storage systems? * XML support in the API? * Having Glance understand what is contained in the disk images by inspecting them on upload? * A Glance dashboard app? Please feel free to expand on any of the above and add any suggestions you have on the future direction of Glance. Your input is truly appreciated. Cheers! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
This is an excellent point, Mark, I'd forgotten about the quota system for user-defined (and nova-ignored) metadata items. That does throw a bit of a wrench into the mix. Perhaps a small edit to the instance_metadata table to add a column for is_user_added that can be used to segregate metadata items that Nova itself uses to track certain things? -jay On Thu, Apr 14, 2011 at 10:01 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: 1) Continue to add fields to the instances table (or compute_nodes table) for these main attributes like cpu_arch, etc. 2) Use the custom key/value table (instance_metadata) to store these attribute names and their values 3) Do both 1) and 2) I've no particular preference here, but if we start adding attributes to instance_metadata that nova code specifically knows about, we need to ensure there is a good way to tell the difference between nova metadata and user metadata. I do not believe key prefixes will provide this difference cleanly. Currently, instance_metadata is for user-provided attributes only and nova is agnostic to their contents. In the api, they are presented as such, and there are quotas on the number of metadata items the user can set. This would all break if we start adding nova metadata without further api and db (small) changes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] distributed and heterogeneous schedulers
We're just running into this problem with distributed scheduler. The problem is Provision 1000 servers ... how do we load balance this across hosts across zones? It busts the current dump the requests into the queue and let the workers feast approach in place currently. I have a plan for it, which I was implementing. Should have a review branch soonish. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Jay Pipes [jaypi...@gmail.com] Sent: Thursday, April 14, 2011 12:29 PM To: Mark Washenberger Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] distributed and heterogeneous schedulers This is an excellent point, Mark, I'd forgotten about the quota system for user-defined (and nova-ignored) metadata items. That does throw a bit of a wrench into the mix. Perhaps a small edit to the instance_metadata table to add a column for is_user_added that can be used to segregate metadata items that Nova itself uses to track certain things? -jay On Thu, Apr 14, 2011 at 10:01 AM, Mark Washenberger mark.washenber...@rackspace.com wrote: 1) Continue to add fields to the instances table (or compute_nodes table) for these main attributes like cpu_arch, etc. 2) Use the custom key/value table (instance_metadata) to store these attribute names and their values 3) Do both 1) and 2) I've no particular preference here, but if we start adding attributes to instance_metadata that nova code specifically knows about, we need to ensure there is a good way to tell the difference between nova metadata and user metadata. I do not believe key prefixes will provide this difference cleanly. Currently, instance_metadata is for user-provided attributes only and nova is agnostic to their contents. In the api, they are presented as such, and there are quotas on the number of metadata items the user can set. This would all break if we start adding nova metadata without further api and db (small) changes. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Summit Talk: Information session on Zones? Any interest?
I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
I am very interested in this topic. From: Sandy Walsh sandy.wa...@rackspace.com Date: Thu, 14 Apr 2011 16:07:58 + To: openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: [Openstack] Summit Talk: Information session on Zones? Any interest? I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
+1 On Thu, Apr 14, 2011 at 12:52 PM, Daniel Salinas imsplit...@gmail.comwrote: I am very interested in this topic. From: Sandy Walsh sandy.wa...@rackspace.com Date: Thu, 14 Apr 2011 16:07:58 + To: openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: [Openstack] Summit Talk: Information session on Zones? Any interest? I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netUnsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Judd Maltin T: 917-882-1270 F: 501-694-7809 A loving heart is never wrong. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
+1 -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin On Apr 14, 2011, at 12:07 PM, Sandy Walsh wrote: I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp smime.p7s Description: S/MIME cryptographic signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
+3 2011/4/15 Chris Behrens chris.behr...@rackspace.com +2 On Apr 14, 2011, at 9:07 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netopenstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] NaaS proposal suggestion
As many of you know there are a few Network as a Service proposals floating around. All of the authors are working to combine them into something we all want to move forward with. Hopefully by the summit we will have one blueprint to rule them all. I would like to make a couple suggestions publicly that I have been mentioning to everyone I talk to about NaaS. 1. NaaS should be optional nova's existing hypervisor only flat and vlan network functionality should stay in nova. You should not need to bring up a separate service to bring up a simple test instance. This will also help us not break nova as we are making rapid code changes. 2. all communication should be via API. NaaS should not write or read directly from Novadb. I have seen many diagrams that have the NaaS writing data directly to novadb. 3. NaaS should be generic enough that other things can consume it. I would love to see Opennebula and Eucalyptus be able to use the Openstack NaaS. I know of a few sites that have both Eucalyptus and Openstack deployed. It would be nice if they could share a NaaS. i would also like to support application calling NaaS to create their own shared network containers. Cheers, Rick Principal Engineer, Cisco Systems signature.asc Description: OpenPGP digital signature ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
I am also interested in this. On Thu, Apr 14, 2011 at 9:15 AM, Edward koko Konetzko konet...@quixoticagony.com wrote: On 04/14/2011 11:07 AM, Sandy Walsh wrote: I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp I would be really interested in a talk on zones, I know a few other people who would be too. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
I'm not sure what you mean by an 'informational session'? That sounds rather didactic - is everything sufficiently agreed for a session of this type? I was under the impression that a lot of the details of zones were not yet agreed, and that a lot of the use cases we would like to see in future were not yet designed in. I would very much appreciate a Current State of Zones presentation that would precede a discussion session on what zones should look like in Diablo and beyond. We want to be sure that our zones design supports our ambitions e.g. federation / bursting. Or is this what you're suggesting? In which case +1 ! Justin On Thu, Apr 14, 2011 at 6:07 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
From: Justin Santa Barbara [jus...@fathomdb.com] I would very much appreciate a Current State of Zones presentation that would precede a discussion session on what zones should look like in Diablo and beyond. We want to be sure that our zones design supports our ambitions e.g. federation / bursting. Or is this what you're suggesting? In which case +1 ! Justin heh, good point. Yes, informative regarding what's in place and discussion around the plan for dist scheduler, which is underway. There is a separate Auth session in the works which will touch on Zones requirements as well. Sorry for making it sounds like it's a done-deal. It's certainly not. :) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] NaaS proposal suggestion
Rick - Agree with everything below. IMO, #3 should apply in general to all OS services (core network, block storage, load balancing, etc.) We want things to work as a suite of services but each service should be independent and deployable by itself. There will obviously by interface standards that will need to be adhered to, but totally doable. The more we can make each OS service the canonical API and automation engine for each IaaS piece, the better. And part of that is making it usable with non-OS services. Erik Sent from my iPhone On Apr 14, 2011, at 1:08 PM, Rick Clark r...@openstack.org wrote: As many of you know there are a few Network as a Service proposals floating around. All of the authors are working to combine them into something we all want to move forward with. Hopefully by the summit we will have one blueprint to rule them all. I would like to make a couple suggestions publicly that I have been mentioning to everyone I talk to about NaaS. 1. NaaS should be optional nova's existing hypervisor only flat and vlan network functionality should stay in nova. You should not need to bring up a separate service to bring up a simple test instance. This will also help us not break nova as we are making rapid code changes. 2. all communication should be via API. NaaS should not write or read directly from Novadb. I have seen many diagrams that have the NaaS writing data directly to novadb. 3. NaaS should be generic enough that other things can consume it. I would love to see Opennebula and Eucalyptus be able to use the Openstack NaaS. I know of a few sites that have both Eucalyptus and Openstack deployed. It would be nice if they could share a NaaS. i would also like to support application calling NaaS to create their own shared network containers. Cheers, Rick Principal Engineer, Cisco Systems ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Summit Talk: Information session on Zones? Any interest?
+1 Enviado desde mi iPad El 14/04/2011, a las 18:07, Sandy Walsh sandy.wa...@rackspace.com escribió: I've been getting a lot of questions about Zones lately. How much interest is there for an informational session on Zones and, I guess, Distributed Scheduler and roadmap? (pending an available slot at the summit ... things are filling up quickly I gather) -S Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
