when services are not exposed I assumed we shall keep the existing mechanism but we should consider if the test has a real added value compared to built-in kubernetes mechanisms (it could be a way to reduce PTL's pain to maintain this kind of tests). If the tests consist in verifying the status of pods, it can be done through existing liveness checks
robot is just a test framework from an integration perspective, I think that it is better to consider ONAP as an isolated System Under Test. So my first option will be to develop more external tests whatever the test framework. Unit tests and functional tests are under the project responsibility. Ideally such tests shall be run on target env - it means an ONAP deployed by OOM (extension of the gating) - but it requires resources. Today we are performing the 2 types of tests in the CI gating and daily chains some are run from outside the cluster (usually as a docker), some from inside (run as a kubernetes job in ONAP namespace) These tests are in the different CI stages - infra-healthcheck - healthcheck - smoke tests in smoke tests you have both coexisting the xtesting docker smoke-usecases-pythonsdk (outside) smoke-usecases-robot (inside) regarding robot healthcheckI think we could keep what we have today, re-use the same robot tests for external tests (like in ingress) then see the delta (test not exposed / value of the healthcheck) and simplify if needed and possible. Let's discuss this topic with Integration team next week. /Morgan ________________________________________ De : Krzysztof Opasiak [[email protected]] Envoyé : mardi 23 juin 2020 20:57 À : RICHOMME Morgan TGI/OLN; [email protected] Cc : Kuzmicki, Krzysztof (Nokia - PL/Wroclaw); [email protected]; DESBUREAUX Sylvain TGI/OLN; [email protected]; [email protected]; [email protected]; FREEMAN, BRIAN D; [email protected]; [email protected]; [email protected]; [email protected] Objet : Re: [ONAP][Guilin][Integration] refactorig of robot pod Hi Morgan, On 23.06.2020 18:01, [email protected] wrote: > Hi > > during the vF2F meeting we exchanged on the refactoring of the robot pod. > > Today the robot pod is installed through a helm chart hosted in > testsuite/oom (https://git.onap.org/testsuite/oom/ > <https://protect2.fireeye.com/url?k=d70f0f1c-8ac3c3b9-d70e8453-0cc47a30d446-3a274fdcd807db27&q=1&u=https%3A%2F%2Fgit.onap.org%2Ftestsuite%2Foom%2F>) > it includes a docker testsuite > (https://nexus3.onap.org/#browse/search=keyword%3Dtestsuite:a5e575898d3470ccbf77bc9afa6ce9ba > <https://protect2.fireeye.com/url?k=dde6076b-802acbce-dde78c24-0cc47a30d446-eba8a422c15ef714&q=1&u=https%3A%2F%2Fnexus3.onap.org%2F%23browse%2Fsearch%3Dkeyword%253Dtestsuite%3Aa5e575898d3470ccbf77bc9afa6ce9ba>) > that contains everything (robot execution env + web server to expose the > robot results) > the associated configmap is referencing only intra-cluster addresses. > During installation, bash scripts are created on the OOM control node to > call robot execution inside the robot pod and run robot test from the > control node. > In parallel, these robot pods have been integrated in xtesting dockers > and are executed during CI as a kubernetes jobs within the ONAP namespace. > > For Guilin, I would like to finalize the change initiated by Daniel > I know that you started working on this topic as part of the ingress > evolution. > I can see several patches around > https://gerrit.onap.org/r/q/project:testsuite/oom+ingress > <https://protect2.fireeye.com/url?k=e0c32958-bd0fe5fd-e0c2a217-0cc47a30d446-9b634c8aec5eaad6&q=1&u=https%3A%2F%2Fgerrit.onap.org%2Fr%2Fq%2Fproject%3Atestsuite%2Foom%2Bingress> > > I assume we should move to this direction > - new helm chart hosted in OOM repository > - review this helm chart to split the robot execution and the results > display (review the different dockers needed (nginx), refactor testsuite) > - run the test from outside (not from the cluster) => change the @ in > the configmap So as a part of this job we would need to decide on the policy that we'll apply to robot tests. As currently robot tests are often not system level test but more like unit tests, there are a lot of services which are not exposed to the outside world but are tested using robot. Do you have any idea how to deal with that? Should robot be used only for external (system-level, user focused) tests or it should do both? > - keep on supporting ingress and legacy (except is ingress is fully > adopted in Guilin) > > I assume we may have the 2 charts in // during a transition phase but > for Guilin the new version shall be available. > ONAP end users are now familiar with the robot pods, so the change shall > be transparent for them. > > Please feel free to comment/amend/criticize/... > > I can plan a slot on this topic next week during the Integration weekly > meeting to see how we can move forward. > > /Morgan > > NB: some ref/jira/gerrit > initial commit of the python 3 based image > https://jira.onap.org/browse/TEST-150 </97660</a></div> <div>move > testsuite and all its repos to python 3: <a href=> > > _________________________________________________________________________________________________________________________ > > Ce message et ses pieces jointes peuvent contenir des informations > confidentielles ou privilegiees et ne doivent donc > pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu > ce message par erreur, veuillez le signaler > a l'expediteur et le detruire ainsi que les pieces jointes. Les messages > electroniques etant susceptibles d'alteration, > Orange decline toute responsabilite si ce message a ete altere, deforme ou > falsifie. Merci. > > This message and its attachments may contain confidential or privileged > information that may be protected by law; > they should not be distributed, used or copied without authorisation. > If you have received this email in error, please notify the sender and delete > this message and its attachments. > As emails may be altered, Orange is not liable for messages that have been > modified, changed or falsified. > Thank you. > -- Krzysztof Opasiak Samsung R&D Institute Poland Samsung Electronics _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21478): https://lists.onap.org/g/onap-discuss/message/21478 Mute This Topic: https://lists.onap.org/mt/75076889/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
