On 7/28/2016 11:20 PM, Ghanshyam Mann wrote:
Yes, I also prefer the approach of capping the tests instead of jobs. But along
with that we might need to make sure the same tests coverage Tempest provides if
min_microversion is set >2.35 in config.
For example, if we cap the tests (calls nova-network) with max_microversion =
2.35 then, we might need to implement/modify those tests to start using neutron
which can be run if config's min_microversion
is set > 2.35.
There are two type of test cases-
1. Test only tests nova-network APIs - Example:
https://github.com/openstack/tempest/tree/master/tempest/api/compute/floating_ips
2. Test testing other scenario using nova-network - Example:
https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_rescue.py
1st case if all ok to cap and leave it skip for >2.35. But for 2nd case, I feel we
should not leave them skip if config's min_microversion > 2.35 which mean leaving
those scenario untested(if min_microversion >2.35).
There are 2 options for 2nd case:
1. Implement duplicate tests by using the neutron APIs - This will be
duplicate tests but needed because of testing nova-network till newton EOL.
2. Or modify those to switch from nova-network to neutron. - If we do not
care about nova-network testing even for stable branches where it is not
deprecated.
I don't like either option, but I'd rather go with #1 for two reasons:
a) Once nova-network is gone these tests wouldn't run ever, so we lose
the coverage.
b) nova-network wasn't deprecated in stable branches so I think we need
to maintain those tests, so 2.1-2.35 are nova-network, 2.36+ are neutron.
The duplication cost probably wouldn't be all that terrible, we could
probably abstract a lot of the common network setup parts away for the
compute API tests for things like creating a floating IP.
Thanks
gmann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Thanks,
Matt Riedemann
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
