Re: [Openstack] Some insight into the number of instances Nova needs to spin up...
Bret Piatt wrote: We can break this down into a list of measureable test cases. Many of these tests aren't just single host system tests. They require an integrated deployment to find and eliminate the bottleneck. I'm certain I'm missing additional items we would want to measure so consider this an off the top of my head incomplete list. Rate of change tests: 1. Maximum rate of change per host machine -- this could be create, delete, migrate, snapshot, backup. 2. Maximum number of host machines per host controller machine that it can sustain at their maximum rate of change. 3. Maximum number of host machines per glance machine that it can sustain at their maximum rate of change. 4. Maximum number of requests per second and total buffer of requests on the message queue per machine. 5. Maximum number of storage volume operations per storage controller machine. Scale tests: 1. Maximum number of VMs on a host machine. 2. Maximum number of VMs per host controller. 3. Maximum number of storage volumes attached to a host machine. 4. Maximum number of storage volumes per storage controller machine. 5. Maximum number of VM records in the cloud database. 6. Maximum number of storage volume records in the cloud database. 7. Maximum number of VM images managed per glance machine. I understand that these maximum numbers will vary greatly depending on what a machine, the size of the test VM image, the network configuration, and many other factors. I propose we get a test bed setup to measure our own CloudMIPS (call it whatever we want) like a BogoMIPS so we can measure the performance of the codebase over time. We can run the tests each weekend on the trunk as of 00:00 GMT Saturday and by tracking it against the code merged each week ensure we're headed in the right direction and if not we'll be consciously aware of which changes impacted the system performance. I'm up for organizing the effort -- finding hardware, DC space, and getting operational support to manage it. Is anyone interested in participating? This is parallel to the effort Jay described at the summit, about doing automated, Hudson-driven, continuous feature and performance testing on trunk or user-submitted branches: https://blueprints.launchpad.net/nova/+spec/testing-hudson-integration We'll need all the hardware we can get to make that dream a reality, so we should unite the efforts rather than fragmenting them. I hope we can work on that for Cactus. -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Some insight into the number of instances Nova needs to spin up...
On Wed, 29 Dec 2010 19:27:09 + Erik Carlin erik.car...@rackspace.com wrote: The 1M host limit still seems reasonable to me. [] In my opinion, such numbers are completely out of whack. Google's Chubby article says that the busiest Chubby has 90,000 clients (not hosts!) and the biggest datacenter has 10,000 systems. They found such numbers pushing the border of unmanageable. Granted they did not use virtualization, but we're talking the number of boxes in both cases. So to reach 1M hosts in a Nova instance you have to have it manage 100 datacenters. There are going to be calls for federation of Novas long before this number is reached. Sustaining a high flap rate is a worthy goal and will have an important practical impact. And having realistic sizing ideas is going to help it. -- Pete ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Some insight into the number of instances Nova needs to spin up...
We know Amazon is highly, highly elastic. While the instances launched per day is impressive, we know that many of those instances have a short life. I see Guy is now teaming up with CloudKick on this report. The EC2 instance ID enables precise measurement of instances launched, and CloudKick provides some quantitative measure of lifetime of instances. Last time I checked, those numbers we're something like 3% of EC2 instances launched via CK were still running (as a point of reference, something like 80% of Rackspace cloud servers were still running). To meet the elasticity demands of EC2, nova would need to support a high change rate of adds/deletes (not to mention state polling, resizes, etc). Is there a nova change rate target as well or just a physical host limit? The 1M host limit still seems reasonable to me. Large scale deployments will break into regions where each region is an independent nova deployment that each has a 1M host limit. Erik On 12/29/10 10:47 AM, Jay Pipes jaypi...@gmail.com wrote: Some insight into the number of instances being spun up *per day* on AWS, *just in the US-East-1 region*: http://www.jackofallclouds.com/2010/12/recounting-ec2/ Avg in 2010: 70,528 instances spun up each day Max: 150,800 instances in a single day Food for thought. What do we think Nova could/can support? I know we are aiming at supporting clouds of 1M physical hosts. Perhaps we need to re-assess? :) /me urges more prioritization of the openstack-ci (continuous integration and performance/stress testing) project in Cactus... -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
