On Thu, Jan 07 2016, Jay Pipes wrote: > OK, I just watched that. Sorry, still don't see the value that Mimic provides > over unit testing the client interfaces and mocking out the HTTP payloads so > you have strict control over the expectations. > > The problem that Glyph noted in the video about the unit test that mocked out > os.chdir to improperly return a value isn't solved whatsoever by Mimic, so I > find it odd that that example was used in discussing the value of Mimic. Bad > mocks are just that: bad mocks. The same false positive failure could easily > be > introduced with a typo in the "metadata injection" that Mimic does to inject > failures into the system. > > Mimic isn't testing anything on the server side at all so I'm not sure why > folks call it "integration testing". It isn't testing the integration of > anything at all. All it enables is multi-language client library testing, and > see my response to Ben, the surface area it introduces for bugs in the test > platform itself in my opinion outweigh the multi-language value it might have.
I completely agree with what Jay points out here. To illustrate with what we do in the Telemetry team: in order test gnocchiclient, we pip install gnocchi¹, configure it briefly and use the client against that freshly installed server. It works very well, and we're sure we test against real data. That's what I would call integration testing. We could also easily test against older version of the Gnocchi server by pip install-ing an older version if we wanted. This has way more value than mocking the whole HTTP server and crossing fingers we did a good job mimicking it. Oh, and it's also actually less work. :) ¹ https://github.com/openstack/python-gnocchiclient/blob/master/setup-tests.sh -- Julien Danjou ;; Free Software hacker ;; https://julien.danjou.info
signature.asc
Description: PGP signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
