Yes, the “issue” is simply related that we all leverage on git master instead of released packages (see OpenStack Requirements’ upper-constraints) during the development cycle. More, this case is a little bit more complex as this function shouldn’t be considered as part of our framework and we shouldn’t expect this obsolete internal method to reused outside.
Cédric De : [email protected] [mailto:[email protected]] De la part de SULLIVAN, BRYAN L (BRYAN L) Envoyé : vendredi 2 février 2018 15:02 À : Manuel Buil; OPNFV-TECH-DISCUSS OPNFV Objet : Re: [opnfv-tech-discuss] Could ways of working with testing frameworks be improved when working with master? Manuel, Apart from the basic issue you are facing being dependency on test frameworks (which can be a convenient tool, until they change out from under you without notice), the basic problem OPNFV faces here is that it needs to, and easily can, provide such notice automatically. For example, when there is a patch that will cause dependencies to break, the easiest case being that some function no longer exists, it should be straightforward to run a scan of all OPNFV repos to check for references to that function. Other cases, e.g. where a function changes in definition/input etc may be harder, but there should also be notice somehow, supported by a similar scan for references to that function. In Models, VES, and Copper, I solve the basic problem you are facing by not depending upon any of the test frameworks/utils, and to the extent possible on none of the installer frameworks/utils either. But in your case I agree the community should support effective notice that something is likely to break. Thanks, Bryan Sullivan | AT&T From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Manuel Buil Sent: Friday, February 02, 2018 4:00 AM To: OPNFV-TECH-DISCUSS OPNFV <[email protected]<mailto:[email protected]>> Subject: [opnfv-tech-discuss] Could ways of working with testing frameworks be improved when working with master? Hi, As you know, the SFC project is currently testing openstack master, that means we need to use functest master in order to consume the lastest patches in the projects we use. Unfortunately, in the last weeks, we wasted quite some time trying to understand why suddenly the tests were not able to run and the cause was a change in the master branch of functest. For example, we are using the function "timethis" from functest_utils, which was removed by this patch: https://github.com/opnfv/functest/commit/c6092cb676363d89f366dc9a416ba6c53eeea33f#diff-b5f06ecfb223c80624f432ef33cf1fdd<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_opnfv_functest_commit_c6092cb676363d89f366dc9a416ba6c53eeea33f-23diff-2Db5f06ecfb223c80624f432ef33cf1fdd&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=j8ivdxqB81FrnWMaB0iNxRfKwI3TQfVMEJhuGfLlZdY&s=RkBPe60Yxqt5Ez4G-fIQK-AzCOMITZZo6hVSQ3jLaZc&e=> and suddenly our tests are not working and we don't know whether there is an alternative. Functest is the framework we use for our tests and ideally, we (SFC project) would like to get some heads up before that change is done, so that we are warned and we don't have to waste time investigating what changed. I guess the same could be applied to other core testing frameworks like Yardstick. However, this is complicated and I am not sure if there is a good solution to achieve that level of communication without impacting the efficiency of funcatest/yardstick/... development. I have some ideas: A) Functest/Yarstick leave the old functionality for a week adding a log saying "This is going to be deprecated, please check this patch: xxxx" B) Add gates in functest/yardstick projects which run tests of their customer projects (as in, SFC is a customer of functest). This way, projects could be warned on time C) Do nothing. Sorry, this is the consequence of consuming functest/yardstick master D) ?? From what I heard in the TSC, Apex is going to join the XCI philosophy and allow working with the tip of master, so it seems to me that more functest and potentially yardstick users are going to hit this problem, that's why I believe it could be a good time to discuss possible solutions. Please don't get me wrong, I am not trying to blame functest (or yardstick), I just want to share the problems we are having with the current ways of working and try to find a solution :) Thanks, Manuel _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
