Re: [openstack-dev] [all] Million level scalability test report from cascading
Hi Rob There is no API request failure under current hardware configuration. If you remove one physical server for NOVA API service or for scheduler service, then API request failure will happen under 1000 concurrency. Similar to Neutron. Before expansion to the current test environment hardware configuration, API request failure was observed. For the system load during the steady-state, because the cascaded OpenStack simulator will make the VM status changed now and then ( and also for volume, port ), these status change will be periodically batch polled and synchronized to the cascading layer. The status change is more frequently than normal steady-state in one OpenStack instance. That's why you see the load is not very low even if there is no API request. For VM operation failure rate, because the cascaded OpenStack is simulator, so it's not measured. The rate is co-related how the model is implemented in the cascaded simulator. === Something wrong with my openstack-dev mail list subscription. I can receive others' mail, but cannot receive all mails related to the current thread. Have to re-subscribe and send the mail again. Best Regards Chaoyi Huang ( joehuang ) On 31 March 2015 at 22:05, joehuang joehu...@huawei.com wrote: Hi, all, During the last cross project meeting[1][2] for the next step of OpenStack cascading solution[3], the conclusion of the meeting is OpenStack isn't ready for the project, and if he want's it ready sooner than later, joehuang needs to help make it ready by working on scaling being coded now, and the scaling is on the first priority for OpenStack community. We just finished the 1 million VMs semi-simulation test report[4] for OpenStack cascading solution, the most interesting findings during the test is, the cascading architecture can support million level ports in Neutron, and also million level VMs in Nova. And the test report also shows that OpenStack cascading solution can manage up to 100k physical hosts without challenge. Some scaling issues were found during the test and listed in the report. The conclusion of the report is: According to the Phase I and Phase II test data analysis, due to the hardware resources limitation, the OpenStack cascading solution with current configuration can supports a maximum of 1 million virtual machines and is capable of handling 500 concurrent API request if L3 (DVR) mode is included or, 1000 concurrent API request if only L2 networking needed. It's up to deployment policy to use OpenStack cascading solution inside one site ( one data center) or multi-sites (multi-data centers), the maximal sites (data centers) supported are 100, i.e., 100 cascaded OpenStack instances. The test report is shared first, let's discuss the next step later. Wow thats beautiful stuff. The next time someone does a report like this, I'd like to suggest some extra metrics to capture. API failure rate: what % of API errors occur. VM failure rate: what % of operations lead to a failed VM (e.g. not deleted on delete, or not started on create, or didn't boot correctly) block device failure rate similarly. Looking in your results, I observe significant load in the steady-state mode for most of the DB's. Thats a little worrying, if as I assume steady-state means 'no new API calls being made'. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Million level scalability test report from cascading
Hi Rob There is no API request failure under current hardware configuration. If you remove one physical server for NOVA API service or for scheduler service, then API request failure will happen under 1000 concurrency. Similar to Neutron. Before expansion to the current test environment hardware configuration, API request failure was observed. For the system load during the steady-state, because the cascaded OpenStack simulator will make the VM status changed now and then ( and also for volume, port ), these status change will be periodically batch polled and synchronized to the cascading layer. The status change is more frequently than normal steady-state in one OpenStack instance. That's why you see the load is not very low even if there is no API request. For VM operation failure rate, because the cascaded OpenStack is simulator, so it's not measured. The rate is co-related how the model is implemented in the cascaded simulator. === I don't know why I can't see my mail and your reply in the OpenStack-dev mail list, but I had thought that the mail has not been sent successfully, but someone else told me he saw the mail. I only found your reply through the mail list achieve : http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg49335.html Best Regards Chaoyi Huang ( joehuang ) On 31 March 2015 at 22:05, joehuang joehu...@huawei.com wrote: Hi, all, During the last cross project meeting[1][2] for the next step of OpenStack cascading solution[3], the conclusion of the meeting is OpenStack isn't ready for the project, and if he want's it ready sooner than later, joehuang needs to help make it ready by working on scaling being coded now, and the scaling is on the first priority for OpenStack community. We just finished the 1 million VMs semi-simulation test report[4] for OpenStack cascading solution, the most interesting findings during the test is, the cascading architecture can support million level ports in Neutron, and also million level VMs in Nova. And the test report also shows that OpenStack cascading solution can manage up to 100k physical hosts without challenge. Some scaling issues were found during the test and listed in the report. The conclusion of the report is: According to the Phase I and Phase II test data analysis, due to the hardware resources limitation, the OpenStack cascading solution with current configuration can supports a maximum of 1 million virtual machines and is capable of handling 500 concurrent API request if L3 (DVR) mode is included or, 1000 concurrent API request if only L2 networking needed. It's up to deployment policy to use OpenStack cascading solution inside one site ( one data center) or multi-sites (multi-data centers), the maximal sites (data centers) supported are 100, i.e., 100 cascaded OpenStack instances. The test report is shared first, let's discuss the next step later. Wow thats beautiful stuff. The next time someone does a report like this, I'd like to suggest some extra metrics to capture. API failure rate: what % of API errors occur. VM failure rate: what % of operations lead to a failed VM (e.g. not deleted on delete, or not started on create, or didn't boot correctly) block device failure rate similarly. Looking in your results, I observe significant load in the steady-state mode for most of the DB's. Thats a little worrying, if as I assume steady-state means 'no new API calls being made'. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Million level scalability test report from cascading
On 31 March 2015 at 22:05, joehuang joehu...@huawei.com wrote: Hi, all, During the last cross project meeting[1][2] for the next step of OpenStack cascading solution[3], the conclusion of the meeting is OpenStack isn't ready for the project, and if he want's it ready sooner than later, joehuang needs to help make it ready by working on scaling being coded now, and the scaling is on the first priority for OpenStack community. We just finished the 1 million VMs semi-simulation test report[4] for OpenStack cascading solution, the most interesting findings during the test is, the cascading architecture can support million level ports in Neutron, and also million level VMs in Nova. And the test report also shows that OpenStack cascading solution can manage up to 100k physical hosts without challenge. Some scaling issues were found during the test and listed in the report. The conclusion of the report is: According to the Phase I and Phase II test data analysis, due to the hardware resources limitation, the OpenStack cascading solution with current configuration can supports a maximum of 1 million virtual machines and is capable of handling 500 concurrent API request if L3 (DVR) mode is included or, 1000 concurrent API request if only L2 networking needed. It's up to deployment policy to use OpenStack cascading solution inside one site ( one data center) or multi-sites (multi-data centers), the maximal sites (data centers) supported are 100, i.e., 100 cascaded OpenStack instances. The test report is shared first, let's discuss the next step later. Wow thats beautiful stuff. The next time someone does a report like this, I'd like to suggest some extra metrics to capture. API failure rate: what % of API errors occur. VM failure rate: what % of operations lead to a failed VM (e.g. not deleted on delete, or not started on create, or didn't boot correctly) block device failure rate similarly. Looking in your results, I observe significant load in the steady-state mode for most of the DB's. Thats a little worrying, if as I assume steady-state means 'no new API calls being made'. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
