Hi Tim, api_check is part of new SNAPS tests (framework developed by Steve and integrated in Danube) - 2 in healthcheck: connection_check and api_check - 1 in smoke: snaps_smoke we integrated them end of next week, it seems that we need to troubleshoot a little bit :) we had a discussion on the category (healtcheck versus smoke)
As the 2 first test suites were short - healthcheck is as you mentioned mainly for gating so it should be reliable and not block installer CI - we put them in healthcheck (they could eventually replace the old one written in bash) but as you say, negative tests must not be in healthcheck (good point, we should include that in our rule for category selection..), I move api_check to smoke (as Tempest) patch => https://gerrit.opnfv.org/gerrit/#/c/25783/1 + Steve in copy /Morgan Le 12/12/2016 à 03:07, Tim Rozet a écrit : > Hi Jose, Morgan, > I noticed in our latest verifies there are some new test cases added called > 'api_check' that are running as part of the healthcheck: > > https://build.opnfv.org/ci/job/functest-apex-apex-verify-master-suite-master/734/console > > Most of these test cases pass, but the negative tests seem to fail like: > test_create_subnet_empty_cidr > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... > FAIL > test_create_subnet_empty_name > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... > FAIL > test_create_subnet_null_cidr > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... > FAIL > test_create_subnet_null_name > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... > FAIL > > Should those be running as part of the healthcheck? If so, can we remove the > negative test cases? I feel like the smoke tests should just be for > valdidating basic functionality of what is deployed works, rather than > negative tests. We still need to figure out why these are failing, but I > don't want it to block our verify CI (maybe turn on in daily?). > > There are a few positive tests that failed: > test_create_network > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsNetworkTests) > ... FAIL > test_create_subnet > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsSubnetTests) ... > FAIL > test_create_port > (snaps.openstack.utils.tests.neutron_utils_tests.NeutronUtilsRouterTests) ... > FAIL > test_create_router_private_network > (snaps.openstack.tests.create_router_tests.CreateRouterSuccessTests) ... ERROR > > I don't know why those failed, but creating networks, subnets, etc all passed > in the previous healtcheck tests. > > Thanks, > > Tim Rozet > Red Hat SDN Team -- Morgan Richomme Orange/ IMT/ OLN/ CNC/ NCA/ SINA Network architect for innovative services Future of the Network community member Open source Orange community manager tel. +33 (0) 296 072 106 mob. +33 (0) 637 753 326 morgan.richo...@orange.com _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
