I do not want robot code spread among different repositories unless there is no other choice.
I think it will hamper re-use of those test cases for cross projects testing which would be benefecial for all. Brian From: Keong Lim <[email protected]> Sent: Sunday, April 07, 2019 9:06 PM To: ROSE, DANIEL V <[email protected]> Cc: FREEMAN, BRIAN D <[email protected]>; FORSYTH, JAMES <[email protected]>; LEFEVRE, CATHERINE <[email protected]>; [email protected]; [email protected] Subject: RE: what is the architectural direction for ONAP testsuite repository? Importance: Low Sensitivity: Confidential Thanks for the explanation Daniel. The “80% reusable” should be a forward-looking estimate (of the 279 to come) rather than a backward-looking estimate (of the 10 already existing). The real goal for the BBS use case test is not the customer, nor the service-subscription. These are only the required stepping stones to get the service-instance with metadata. So rather than taking another shortcut to reach metadata objects quickly, the more valuable way is to make the customer->service-subscription composition work as easily as the service-subscription->service-instance composition, which can then be reused for the service-instance->metadata composition and all other sub-objects. I’m sorry if I read it wrongly, but as I understand it a -1 review means “please change this before it can be accepted” whereas a -2 review means “never going to happen”. As such, receiving a -2 review prompts me to ask what is the correct way forward. Is this not the meaning that was intended by the -2 review? Regarding a new “get customer” method, it might be nice to write it only once and forget about it, but unfortunately, if we are testing regressions of schema changes, then we would end up writing it once per schema change (and potentially once per API version regardless of schema change). We need to be prepared for the second, third, … and subsequent versions of these methods. How will the testsuites repo cope with all of that? It is definitely a change of direction compared to the current set of scripts and templates. I do expect to compose these methods together with the many sub-object relationships. If this composition cannot be worked out properly, then we will also end up duplicating many versions of sub-objects like p-interface, l-interface, lag-interface which are dependent on several objects, e.g. generic-vnf->l-interface, newvce->l-interface, p-interface->l-interface, lag-interface->l-interface, l-interface->l-interface. For AAI tests, this composition should be done in the test cases, so they can be manipulated in the regression tests. For demo scenarios, perhaps these should be done in a new layer of the keywords for ease-of-use instead. This area has not been explored properly yet. The existing keywords and templates may have been a short-term quick fix to get the demo up and running with a small graph of objects. While the demo code may improve and change in future, for now I would explore a parallel construction until it can be easily cut over. If that parallel construction doesn’t work out, then it will be easy to discard and try again without affecting the existing code path for the demo. Keong From: ROSE, DANIEL V [mailto:[email protected]] Sent: Friday, April 5, 2019 11:13 PM To: Keong Lim <[email protected]<mailto:[email protected]>> Cc: FREEMAN, BRIAN D <[email protected]<mailto:[email protected]>>; FORSYTH, JAMES <[email protected]<mailto:[email protected]>>; LEFEVRE, CATHERINE <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]> Subject: Re: what is the architectural direction for ONAP testsuite repository? Sensitivity: Confidential I think a robot tag of csit is fine, as is putting tests that are just for for aai under aai folder which you did so good. The main reason i -2 the code is also not because of timeline but because of the 80% is not reusable even within other csit test cases. I would be extremely happy to see 279 models tested in robot but not as excited if each one comes with its own get customer method 😊 That said what we have existing in resources is certainly not anything i will claim is a perfect architecture so we can certainly enhance it to meet the needs of aai. It seems in your email you mostly have an issue with optional fields and relationship lists? If so sounds like the problem is the rigidness of the templating we have (which admittedly i built in about three hours) and is something we can look at. Also the version 11 is probably something that should be upgraded also. On Apr 5, 2019, at 2:53 AM, Keong Lim <[email protected]<mailto:[email protected]>> wrote: Hi Brian, Happy to take advice on filenaming conventions. I suspect there could end up being many variations, i.e. one per API version and use case scenario. I know you made some edits for INT-896 recently and I don’t want to get in the way of that High priority bug maintenance. Is the strictness of the review due to the looming M4 code freeze milestone? If the testsuite code is frozen at M4, then perhaps it is better for AAI to commit robot test code to its own repositories, as new tests and documentation should be exempted from the code freeze. There are approx. 279 elements (or approx. 140 if you don’t count the container types) in the AAI schema of which maybe 10 are covered by robot testsuites at the moment, so there’s plenty more commits on the way. Look forward to more detailed reviews and suggested solutions. Thanks, Keong From: FREEMAN, BRIAN D [mailto:[email protected]] Sent: Friday, April 5, 2019 3:09 PM To: Keong Lim <[email protected]<mailto:[email protected]>>; FORSYTH, JAMES <[email protected]<mailto:[email protected]>>; ROSE, DANIEL V <[email protected]<mailto:[email protected]>>; LEFEVRE, CATHERINE <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Cc: [email protected]<mailto:[email protected]> Subject: Re: what is the architectural direction for ONAP testsuite repository? Sensitivity: Confidential I will review and give feedback. At a quick glance i see hard coded values like bbs and duplication of existing keywords that will be confusing Get Customer I think this is solvable but we do to look for opportunities to reuse and avoid confusion. I also think we might want to name scripts as csit_ or something to help clarify. Finally the bbs robot seems use case specific instead of a reusable regression but i haven't look closely since i am traveling. Btian -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16498): https://lists.onap.org/g/onap-discuss/message/16498 Mute This Topic: https://lists.onap.org/mt/30919602/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
