Hi Brian thanks for the quick answer some answers & ... questions in the mail ..
/Morgan Le jeudi 23 août 2018 à 12:56 +0000, FREEMAN, BRIAN D a écrit : Morgan, Jenkins jobs for daily installation and regression tests are off the jenkins/grafana site that Gary runs. http://onapci.org/jenkins/ http://onapci.org/grafana/ http://onapci.org/grafana/d/8cGRqBOmz/daily-summary?orgId=1 That is what I had in mind but I did not find the link from the wiki. is there any reason it is not referenced? if no objection , I could add it to the wiki integration page My first reflex was to go to https://jenkins.onap.org/view/integration/ but onapci.org is much clearer from an integration perspective I am not sure to fully understand why there are 2 jenkins dealing with integration. More generally I also remember that there is a dedicate CI chains for OOM. The CI/CD governance is not fully clear. On onapci jenkins, does the daily jobs deal with the Master branch (Casablanca candidate)? is there any jobs using Beijing (to check the stability) I can see that healthcheck tests are performed at the end of the installation (with heat & oom installer). For the moment, none of the healthcheck test reached 100% (which is logic for a master branch at M3 milestone), is it why the VNF tests are not triggered? As far as I remember vFW was integrated in CI chains for Beijing It would make sense to keep all the supported & automated VNF E2E tests in the CI chains. We could start the discussion even before ONS. sure.. I would encourage us to look at incorporating testing into the testsuite project rather than a new project but I may be mis-understanding what you mean by xtesting. I agree it is always challenging, as creation - at least in first step - looks quicker (you are master on board) than joining existing projects due to the learning curve and the technical debt. but one the most important community added value deals with the capability to avoid that. regarding Orange contribution in this area, the problem is more with onap-tests than xtesting. xtesting does not create any use case, it is a docker launcher tool consuming existing testsuites from upstream test projects using different languages/technologies(Robot, bash, python, junit,..). the python onap-tests framework do things similar to what Robot does (onboarding, instantiation, openstack resource creation check). we initiated it mainly to learn (creation of class per ONAP module a little bit like the openstack python clients per Openstack module - but not so well written...) we were not so familiar with robot and, as, when it becomes complex, robot (which is written in python) calls python classes, we did it this way. But we could probably reuse all the robot capabilities to run the vIMS or vAAA We can see that some of the python classes called from robot scripts (hosted in python-testing-utils) are not very different (and probably better) to what we did. regarding xtesting, we are using it to unify the way we launch tests from different projects (infra/kubernetes, infra/openstack, onap/healthcheck, onap/vvp, ..) in different CI/CD chains (jenkins, gitlab). We use it for our infra and for ONAP. for the infra, you can see some exemples from OPNFV CI/CD chains - kubernetes infra check (test suite from kubernetes community): https://build.opnfv.org/ci/view/functest/job/functest-compass-baremetal-daily-master/249/console - openstack infra check (testsuites from different sources:upstream openstack (rally, tempest, shaker, vmtp, ...), OPNFV feature project or test project own dev including VNF lifecycle tests on vIMS, vCPE): https://build.opnfv.org/ci/view/functest/job/functest-apex-baremetal-daily-master/225/console on ONAP we use xtesting to - run the existing Robotframework => docker pull morganrol/xtesting-onap-robot - run ice validator towards referenced heat templates from official ONAP repo => docker pull morganrol/xtesting-onap-vnfpkgcheck [cid:[email protected]] One more time it does not create new testcase, it just "dockerizes" the launch and the way to declare the cases, get and export the results. the test execution is independent from the SUT and cleaned once tests are executed the docker creation shall be triggered each time one of the heat template and/or the code of the ice_validator is modified Installation of ONAP is generally in the demo or integration repos (demo is more for the test vnfs going forward) . OK it is sometimes not fully crystal clear but I do not have all the history in integration/test I can find a directory vcpe https://git.onap.org/integration/tree/test/vcpe there is also several vCPE directories under demo https://git.onap.org/demo/tree/vnfs/vCPE https://git.onap.org/demo/tree/heat/vCPE in integration, there are stuff related to the tests, the labs, the CI, lots of tools. it is not always to see the link between the different folders. would it make sense to create subrepo per VNF rather than a subdirectories in several repos? Testing is in testsuite and has end to end tests as well as domain focused tests for onboarding, instantiation and closed loop The testsuite (+ associated repo heatbridge, properties, python-testing-utils) repo is easier to understand we could imagine to propose for review the little python framework code in the python-testing-utils if it makes sense As you indicated we break up the problem into 3 domains: Demo project -> test vnfs and miscellaneous scripts needed to run demo’s at events. This is also where we have the scripts to build images from opensource tools (e.g. Intel’s vCPE VNFs) Integraiton project -> resources/artifacts supporting the instllation of ONAP via OOM and HEAT – source for environment specific files to drive jenkins CI etec. th Supporting the installation is a huge task would it not be clearer to have subrepos? integration/installer/oom integration/installer/heat and integration/ci for ci related jjb/scripts/.. Testsuite project -> automated testing of ONAP in its role of onboard, instantiate and control loop using the ONAP instance deployed by Integration and VNFs in the demo project . Testsuite includes mostly robot framework tests but also python based tests/interfaces are allowed/encouraged where they make sense. OK I will have a look how we could propose the python code in this repo. in our project (https://gitlab.com/Orange-OpenSource/onap-tests) we also reference postman collection for manual testing, we could also integrate in testsuite/postman? Brian _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#12050): https://lists.onap.org/g/onap-discuss/message/12050 Mute This Topic: https://lists.onap.org/mt/24929469/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
