Hi Jimmy, Daniel, Catherine, Stephen, Brian, Based on Catherine's presentation at https://wiki.onap.org/display/DW/PTL+2019-02-19 the AAI team has started to add more robot testcases into the testsuites repository for CSIT-style regression testing of each releases' use cases. See also https://jira.onap.org/browse/AAI-2184 and https://jira.onap.org/browse/AAI-2208
For AAI, that means wanting to: - detect the presence/absence of schema elements and EdgeRule relationships as they change from version (n) to (n+1) - confirming that these changes stay functional for subsequent releases - being able to manipulate individual objects/sub-objects and relationships in a flexible and general-purpose manner - (eventually) achieve 100% coverage of schema elements and relationships - (perhaps) testing performance of AAI services across the board (some problem use cases have been identified in https://wiki.onap.org/display/DW/2019-04-04+AAI+Developers+Meeting ) The current templates and robot keywords in testsuite repository seems to be focussed on the demo scenario and healthchecks, to the point of having many hard-coded values, sub-objects and relationships, as well as being stuck on API v11 schema and minimal coverage of the bare necessities. In the first gerrit review https://gerrit.onap.org/r/#/c/79583/ there was a bit of discussion as to whether the testsuites repository is the right place to put such things. The feeling was that the tests are only run on the latest release and don't need to test for regressions. But eventually it was accepted and merged. In the most recent gerrit review https://gerrit.onap.org/r/#/c/84195/ we get a -2 on working code (that also doesn't break existing code) because it didn't reuse hard-coded demo testcases. We don't want to change the demo, but I think the demo code should be refactored so that demo-specific parts are separate from general-purpose reusable common base code. For refactoring to happen, we still need more foundations that allow for manipulating sub-objects and relationship-lists individually, as well as being able to compose the relationships between schema elements in the same way they are modelled in the AAI services. We need to avoid the problems inherent in number of REST URL combinations as enumerated in the swagger-generated API documents, while still being able to test every part of the schema. There are still more steps to be taken. We are still just setting the stage. There are no backwards steps in these commits. We want to be a help rather than a hindrance to the ONAP projects. We do not want to be swimming against the current. Right now, feels like there's a fraction too much friction... So the questions are: - what is the architectural direction for testsuite repository? - Is the testsuites/aai sub-directory a place that the AAI team can use as required? - Should these AAI-specific CSIT-style robot testcases go into an AAI repository instead of the testsuite repository? - Do other ONAP projects need/want to share in the robot scripts that AAI team is writing? Thanks, Keong -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16478): https://lists.onap.org/g/onap-discuss/message/16478 Mute This Topic: https://lists.onap.org/mt/30919602/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
