On Feb 24, 2015, at 9:47, Sean Dague <s...@dague.net> wrote:

> I'm happy if there are other theories about how we do these things,
> being the first functional test in the python-novaclient tree that
> creates and destroys real resources, there isn't an established pattern
> yet. But I think doing all CLI calls in CLI tests is actually really
> cumbersome, especially in the amount of output parsing code needed if
> you are going to setup any complicated resource structure.

I think I'm in agreement with the pattern you describe.

I imagine having a set of functional tests for the API, that don't do any CLI 
calls at all. With that we test that the API works properly. Then have a 
separate set for the CLI, which only calls CLI for the command being tested, 
everything else to set up and tear down the test done by API calls. This would 
be done with the rationale that because the entire API functionality is tested 
separately, we can safely use it for setup/teardown with the intent to isolate 
the CLI test to the command being tested and avoid introducing side effects 
from the CLI commands.

But I suppose one could make the same argument for using CLI everywhere (if 
they are all tested, they can also be trusted not to introduce side effects). I 
tend to favor using the API because it's the most bare bones setup/teardown we 
could use. At the same time I understand the idea of performing an entire test 
using the CLI, as a way of replicating the experience a real user might have 
using the CLI, from start to end. I don't think I feel strongly either way.

For the --poll stuff, I agree the API should have it and the CLI uses it. And 
with and without "poll" functionality should be tested separately, API and CLI.

melanie (melwitt)

Attachment: signature.asc
Description: Message signed with OpenPGP using GPGMail

OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Reply via email to