On Fri, Aug 15, 2014 at 01:57:29AM +0400, Boris Pavlovic wrote: > Matt, > >> One thing did just occur to me while writing this though it's probably worth > > investigating splitting out the stress test framework as an external > > tool/project after we start work on the tempest library.  > > > > I fully agree with the fact that stress testing doesn't belong to Tempest. > > This current thread is all about this aspect and all my arguments, related > to splitting Rally and merging it to tempest are related to this. > > > Could you please elaborate, why instead of ready solution "Rally" that has > community and is aligned with OpenStack and OpenStack processes you are > going to create from scratch similar solution?
This is the same issue which was brought up on your subunit2sql thread  and has come up several times already in this thread. Rally doesn't work as individual components, you need to use the whole tool chain to do anything with it. There are pieces of Rally which would be very useful in conjunction with all the other projects we have in the gating workflow. But, because Rally has decided to duplicate much of what we already have and then tie everything together in a monolithic toolchain we can't use these pieces which are useful by itself. This doesn't work with the workflow we already have, it also ignores the split in functionality we currently have and are working on improving. Instead of rewriting all the details again just look at  they elaborate on all of this already. You're also ignoring that by splitting out the stress test framework we'd be doing the exact thing you're so opposed to doing in rally. Which is splitting out existing functionality from a larger more complex project into smaller more purpose built consumable chunks that work together to build a modular pipeline, which can be used in different configurations to suit the application. It's also not being created from scratch, the stress framework already exists and has for quite some time. (it pre-dates Rally) We would just be breaking it off into a separate repo to make the split in functionality more clear. It is essentially a separate tool already, it just lives in the tempest tree which I feel is a source of some confusion around it. > > I really don't see any reasons why we need to duplicate existing and > working solution and can't just work together on Rally? So I have to point out the double standard with this statement. You're ignoring all the functionality that Rally has already duplicated. For example, the fact that tempest exists as a battery of tests which is being slowly duplicated in Rally, or the stress test framework, which is functionality that was more or less completely duplicated in Rally. You seem to be under the mistaken impression that by continuing to improve things on QA program projects we're duplicating Rally. Which is honestly something I'm having a hard time understanding. I especially do not see the value in working on improvements to tempest and the existing workflow by replacing everything with Rally. -------- So I think there's been continued confusion around exactly how all the projects work together in the QA program. So I wrote a blog post giving a high level overview here: http://blog.kortar.org/?p=4 Now if Rally is willing to work with us it would really be awesome since a lot of the work has already been done by them. But we would need to rework Rally so the bits which don't already exist in the QA program are interoperable with what we have now and can be used independently. Duplicated functionality would need to be removed, and we would need to add the improvements in Rally on existing functionality back into the projects, where they really belong. However, as I've been continually saying in this thread Rally in it's current form doesn't work with our current model, nor does it work with the vision, at least I have, for how things should be in the future. Now I'm sure someone is going to see the flow diagrams on my blog and see it as a challenge to explain how Rally does it better today, or something along those lines. Please don't, because honestly it's irrelevant, these are just ideas in my head, and where I want to help steer the QA program as long as I'm working on it. (at least as of today) I fully expect things to evolve and grow more organically resulting in something that will be completely different from that. Also, I'm really done with this thread, I've outlined my stance repeatedly and tried my best to explain my position as clearly as I can. At this point I have nothing else to add. I view the burden as being fully on the Rally team to decide whether they want to start working with the QA program towards integrating Rally into the QA program, (the steps Sean outlined in  are a good start) or remain a separate external project. (or I guess unless the TC mandates something else) One last comment, I do want to apologize in advance if any of my words seem harsh or antagonistic, that's really not my intent. -Matt Treinish  http://lists.openstack.org/pipermail/openstack-dev/2014-August/041815.html  http://lists.openstack.org/pipermail/openstack-dev/2014-August/042069.html  http://lists.openstack.org/pipermail/openstack-dev/2014-August/042269.html <snip>
Description: PGP signature
_______________________________________________ OpenStack-dev mailing list OpenStackfirstname.lastname@example.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev