On 03/01/2017 07:35 AM, Jordan Pittier wrote: > > > On Wed, Mar 1, 2017 at 3:57 AM, Ghanshyam Mann <ghanshyamm...@gmail.com > <mailto:ghanshyamm...@gmail.com>> wrote: > > Doing gradual refactoring and fixing plugins time to time needs lot > of wait and sync. > > That needs: > 1. Plugins to switch from current method usage. Plugins to have some > other function or same copy paste code what current scenario base > class has. > 2. Tempest patch to wait for plugin fix. > 3. Plugins to switch back to stable interface once Tempest going to > provide those. > > This needs lot of sync between tempest and plugins and we have to > wait for tempest refactoring patch for long. > > To make it more efficient, how about this: > 1. Keep the scenario manger copy in tempest as it is. for plugins > usage only. > > Given that the refactoring effort "started" a year ago, at the current > speed it may take 2 years to complete. In the mean time we will have a > massive code duplication and a maintenance burden. > > 2. Start refactoring the scenario framework by adding more and more > helper function under /common or lib. > > Starting a "framework" (each time I see that word, I have a bad feeling) > from scratch without users and usage is very very difficult. How do we > know what we need in that framework and what will be actually used in > tests ?
I'm with Jordan, honestly, common functions and inheritance aren't going to help anyway. My feeling is that the only way to sanely take things forward is to start pivoting towards fixtures. And basically error if inheritance is used in tests. The functions on base classes is what is causing all the pain of rewind, because it's not a contract, but it's also magically there, so people try to use it. This would mean that every test would have more setUp with a bunch of explicit fixtures, but that's fine, it's a little more time thinking through the writing of the tests, and a lot easier to read them to know what is going on, and to delete them later. Basically get rid of all class/superclass magic. > The effort was called scenario refactoring and I think that's what we > should do. We should not do "start from scratch scenarios" or "copy all > the code and see what happens". > > There's no problem with plugins. We committed to have a stable interface > which is documented and agreed upon. It's clearly written > here > https://docs.openstack.org/developer/tempest/plugin.html#stable-tempest-apis-plugins-may-use > The rest is private, for Tempest internal use. If Tempest cores disagree > with that, then we should first of all put a spec and rewrite that > document. It does seem like there is a tough love moment coming about the fact that APIs were consumed that were specifically out of bounds. Either the team commits to them inbounds, and there is the contract, or moves forward under the existing contract. Though, I expect the issue is that because all the Tempest tests work in this super class way, this is the only stuff people understood and copy pasted. So it does feel like there has to be at least a couple of instances in the Tempest tree of how the team *wants* other projects to write tests that they can point people to. -Sean -- Sean Dague http://dague.net __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
