On Wed, Jul 3, 2013 at 6:19 PM, Christopher Armstrong < [email protected]> wrote:
> On Tue, Jul 2, 2013 at 11:38 PM, Robert Collins > <[email protected]> wrote: > > Radix points out I missed the naunce that you're targeting the users > > of python-novaclient, for instance, rather than python-novaclient's > > own tests. > > > > > > On 3 July 2013 16:29, Robert Collins <[email protected]> wrote: > > > >>> What I'd like is for each client library, in addition to the actual > >>> implementation, is that they ship a fake, in-memory, version of the > API. The > >>> fake implementations should take the same arguments, have the same > return > >>> values, raise the same exceptions, and otherwise be identical, besides > the > >>> fact > >>> that they are entirely in memory and never make network requests. > >> > >> So, +1 on shipping a fake reference copy of the API. > >> > >> -1 on shipping it in the client. > >> > >> The server that defines the API should have two implementations - the > >> production one, and a testing fake. The server tests should exercise > >> *both* code paths [e.g. using testscenarios] to ensure there is no > >> skew between them. > >> > >> Then the client tests can be fast and efficient but not subject to > >> implementation skew between fake and prod implementations. > >> > >> Back on Launchpad I designed a similar thing, but with language > >> neutrality as a goal : > >> > https://dev.launchpad.net/ArchitectureGuide/ServicesRequirements#Test_fake > >> > >> And in fact, I think that that design would work well here, because we > >> have multiple language bindings - Python, Ruby, PHP, Java, Go etc, and > >> all of them will benefit from a low(ms or less)-latency test fake. > > > > So taking the aspect I missed into account I'm much happier with the > > idea of shipping a fake in the client, but... AFAICT many of our > > client behaviours are only well defined in the presence of a server > > anyhow. > > > > So it seems to me that a fast server fake can be used in tests of > > python-novaclient, *and* in tests of code using python-novaclient > > (including for instance, heat itself), and we get to write it just > > once per server, rather than once per server per language binding. > > > > -Rob > > > I want to make sure I understond you. Let's say I have a program named > cool-cloud-tool, and it uses python-novaclient, python-keystoneclient, > and three other clients for OpenStack services. You're suggesting that > its test suite should start up instances of all those OpenStack > services with in-memory or otherwise localized backends, and > communicate with them using standard python-*client functionality? > > I can imagine that being a useful thing, if it's very easy to do, and > won't increase my test execution time too much. > The easiest way to do this would be to set up a devstack instance to run your tests against, but that would increase test execution time by at least a few minutes. So although something like this would not work well for unit tests it could be useful for integration tests. Also I do not know of anyone using this for this type of testing yet, but someone always has to be first. > > -- > IRC: radix > Christopher Armstrong > Rackspace > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
