Alex Brett created CLOUDSTACK-7307:
--------------------------------------
Summary: [Automation] Ability to instruct nosetests not to run
tests which require the simulator
Key: CLOUDSTACK-7307
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7307
Project: CloudStack
Issue Type: Improvement
Security Level: Public (Anyone can view this level - this is the default.)
Components: Automation
Affects Versions: 4.5.0
Reporter: Alex Brett
Assignee: Alex Brett
There are a number of Marvin tests which only work if using the simulator, an
example being test_deploy_vm_start_failure in
test/integration/smoke/misc/test_deploy_vm.py
When running tests via nosetests, we currently use a combination of the 'tags'
and 'required_hardware' attributes to select the right tests to run. For
example a KVM advanced zone BVT would run nosetests with {{-a tags=advanced}},
while a simulator test would do {{-a tags=advanced,required_hardware=false}}.
An attempt at solving this issue has been made by setting the required_hardware
attribute to "simulator only", e.g.:
{noformat}
@attr(tags = ['advanced'], required_hardware="simulator only")
def test_deploy_vm_start_failure(self):
{noformat}
Unfortunately this is not practical from nosetests, as you can't do e.g. {{-a
required_hardware!='simulator only'}}, as nosetests does not support this. The
only way now to identify all appropriate tests would be to run it with
something like {{-a tags=advanced,!required_hardware -a
tags=advanced,required_hardware=false -a
tags=advanced,required_hardware=true}}. This is both confusing, and potentially
error prone as if someone adds an additional value to required_hardware,
nosetests will miss it.
In theory it is possible to achieve something using the {{-A}} argument to
nosetests, however experimenting here shows that it would still end up being
very confusing.
I believe the solution is to add a new attribute "simulator_only", at which
point a typical advanced zone BVT could be run with just {{-a
tags=advanced,!simulator_only}}.
I've prepared a patch which adds this attribute.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
