----- Original Message ----- From: "Thierry Carrez" <thie...@openstack.org> To: "OpenStack Development Mailing List" <openstack-dev@lists.openstack.org> Sent: Wednesday, August 6, 2014 4:00:35 PM Subject: [openstack-dev] Which program for Rally
1. Rally as an essential QA tool Performance testing (and especially performance regression testing) is an essential QA function, and a feature that Rally provides. If the QA team is happy to use Rally to fill that function, then Rally can obviously be adopted by the (already-existing) QA program. That said, that would put Rally under the authority of the QA PTL, and that raises a few questions due to the current architecture of Rally, which is more product-oriented. There needs to be further discussion between the QA core team and the Rally team to see how that could work and if that option would be acceptable for both sides. I want to share an use case of Rally for performance bench-marking. I use Rally to benchmark Keystone performance. I can easily get results for comparison between different configurations, openstack distributions etc. Here is a sample result :- https://github.com/stackforge/rally/blob/master/doc/user_stories/keystone/authenticate.rst IMO Rally can play a essential role in performance regression testing. Regards, Neependra Khare Performance Engineering @ Red Hat 2. Rally as an essential operator tool Regular benchmarking of OpenStack deployments is a best practice for cloud operators, and a feature that Rally provides. With a bit of a stretch, we could consider that benchmarking is essential to the completion of the OpenStack project mission. That program could one day evolve to include more such "operations best practices" tools. In addition to the slight stretch already mentioned, one concern here is that we still want to have performance testing in QA (which is clearly essential to the production of "OpenStack"). Letting Rally primarily be an operational tool might make that outcome more difficult. 3. Let Rally be a product on top of OpenStack The last option is to not have Rally in any program, and not consider it *essential* to the production of the "OpenStack" integrated release or the completion of the OpenStack project mission. Rally can happily exist as an operator tool on top of OpenStack. It is built as a monolithic product: that approach works very well for external complementary solutions... Also be more integrated in OpenStack or part of the OpenStack programs might come at a cost (slicing some functionality out of rally to make it more a framework and less a product) that might not be what its authors want. Let's explore each option to see which ones are viable, and the pros and cons of each. -- Thierry Carrez (ttx) _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
