Most of your wishlist items are in the VNFSDK roadmap. #3 on your list is already part of VNFSDK (dovetail-integration repo).
Regarding #2, we have been focusing on packaging tests, but recently updated the functional tests (functest repo) in Beijing. Lifecycle tests have been dormant since OPEN-O, but we can resurrect as soon as we have resources. One prerequisite is a small test version of ONAP (ONAP-lite?). Ideally, we should be able to quickly instantiate it in a cloud environment, run our tests, and tear it down. We should talk about your Python framework. We're cleaning up our test engine this release to make it easier to run Java, Python, or shell tests. Prior to now, we had a number of test engines coming in from different sources, using different languages. I expect by the end of Casablanca to have a stable test framework so we can focus more on the tests in Dublin. I will be at ONS, and happy to meet while we're there. Chris From: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, August 23, 2018 at 5:33 AM To: Yunxia Chen <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, Chris Donley <[email protected]<mailto:[email protected]>>, Gary Wu <[email protected]<mailto:[email protected]>>, "Yang Xu (Yang, Fixed Network)" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Cc: DEBEAU Eric IMT/OLN <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: [ONAP] E2E VNF functional testing Hi, a topic was dedicated to vCPE automation during Integration meeting today. I think that E2E VNF testing automation is key to stabilize ONAP For me automation means the ability to - check packaging - onboard the model - instantiate the VNF - check that resources are propely created on the target infrastructure - perform functional tests (e.g. SIP tests on a vIMS VNF) - clean the resources - report the results Automation shall be replicable on any ONAP platform without big effort except configuration file changes It means decoupling use cases/ONAP installation (no hardcoded values in ONAP installation, no lab manual specific configuration) Proprietary VNFs due to licensing models are very hard to integrate in CI/CD chains (which shall not prevent ambitious use cases to integrate such VNFs) It is clearly challenging As far as I know there are currently several initiatives aiming to provide ONAP VNF E2E automatic testing 1) integration project with the different cases including vCPE (integration team) https://git.onap.org/testsuite/tree/robot https://git.onap.org/demo/tree/ https://git.onap.org/integration/tree/test Using Robot you can already onboard and instantiate several VNFs (vLB, vFW, vVG, vCPE): https://git.onap.org/testsuite/tree/robot/testsuites/model-distribution.robot Some tests were integrated in CI and even use for some robustnes tests (I did not find the jenkins url corresponding to daily runs of there testcases) 2) VNFSDK/VPP as part of VNF certification program (contact C.Donley) information shared during LFN C&C meeting - the idea is to work on a verification program for VNF as the one initiated on the infrastructure by OPNFV VNFs have been announced for September only packaging check are planned for Casablanca as a first step but onboarding/lifecycle tests/functional tests are mentioned in the roadmap dovetail framework is mentioned as the tooling to launch the tests 3) Amdocs demo shared during last ONS summit (contact: Moshe Hoadley) It is also linked to 2) as a poc of ONAP Life cycle testing through dovetail https://wiki.onap.org/display/DW/VNFTEST+integration+with+DOVETAIL?preview=/28377754/28377756/onap-opnfv-collaboration-demo-21_march_2018.mp4 I did not find de code associated to the demo 4) OTF - Open Test Framework (contact: Kevin Wan) https://wiki.onap.org/display/DW/OTF+-+Open+Test+Framework project in progress / still no official repo 5) Light python framework shared during last ONS (contact: Morgan Richomme) This is a python framework allowing to onboard and instantiate VNFs. It can be used with Clearwater vIMS, vMRF (proprietary), vAAA (freeradius), ONAP vFW The code is available here: https://gitlab.com/Orange-OpenSource/onap-tests The plan is to integrate it in xtesting project (as robot healthcheck tests have been integrated) xtesting (https://xtesting.readthedocs.io/en/latest/) is a light framework to harmonize the way of launching the tests (launch, get results, report results), supporting Robot, python, bash, junit based tests. it is used in OPNFV and easy to integrate in CI chains (we use it internally from our gitlab pipelines). 6) Postman collection can be used for automation 7) ... As we can see there are several solutions adressing the same goals It probably would make sense to see if possible synergies are possible Would it make sense to plan a f2f meeting during next ONS summit? /Morgan _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12047): https://lists.onap.org/g/onap-discuss/message/12047 Mute This Topic: https://lists.onap.org/mt/24929469/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
