No, “blocking innovation” was simply related to a possible inability to change 
our internal methods (timethis()) which are out of our Framework.
You highlighted a real problem: we need to remove obsolete utils which don’t 
meet our criteria and may be possibly reused by others.
Cleaning our tree could create side effects in OPNFV projects even if we mostly 
double check anyway.

Yes we should have warned about the next end user changes: TEST_DB_URL and 
creds (and we previously removed prepare_env).
For the previous release, we have simply updated a wiki page (Running Alpine 
container) for all the changes from an end-users POV.
https://wiki.opnfv.org/display/functest/Run+Alpine+Functest+containers

Why not creating a new wiki page for master as well.
But I consider gerrit as the best place simply because it would be delayed by 
mails or by wiki pages.
That may be considered as the cost to select master (it’s also the key 
challenges of XCI ; for instance odl mechanism driver and ODL neutron API may 
be temporarily incompatible if both are master).

The issue is raised because we simply updated Functest and Releng jobs to avoid 
breaking gate but that’s true that we forgot releng-xci.
But such changes rarely happened: here we are integrating the first Kubernetes 
testcases (creds) and we are preparing Xtesting.

Have a nice weekend,
Cédric

De : Manuel Buil [mailto:mb...@suse.com]
Envoyé : vendredi 2 février 2018 16:25
À : OLLIVIER Cédric IMT/OLN; OPNFV-TECH-DISCUSS OPNFV
Objet : Re: [opnfv-tech-discuss] Could ways of working with testing frameworks 
be improved when working with master?

Hi Cedric,

I probably expressed myself wrongly since you understood that I intent to block 
innovation and host SFC functions in functest but that is not my intention. My 
objective with this mail is to check if there exists (perhaps not), an 
efficient way for testing frameworks to communicate in advance important 
changes which will break things in the projects using those frameworks. I have 
a very good example: functest warned us during the Euphrates cycle that 
openstack_utils was not going to be maintained anymore and that we should stop 
using that and migrate to an alternative during the Fraser development cycle. 
You guys even suggested us an alternative (SNAPs). That was really helpful and 
we were able to plan the task accordingly in our SFC backlog and right now we 
are 100% compatible with SNAPs. Note that as far as I know, you haven't removed 
the openstack_utils yet (or at least we were able to use it while doing the 
migration).

However, during the last weeks, some functest changes broke our testing (apart 
from that function I mentioned as an example):

* It seems like the openrc file changed the name in the functest container and 
/home/opnfv/functest/conf/openstack.creds does not work anymore
* TEST_DB_URL is now an environment variable and cannot be consumed from the 
functest config file

To understand how to fix those, we had to go into the IRC channel and ping you. 
If this is only SFC guys doing it, I guess we could continue with option C, but 
if now more users start to work on master (XCI and APEX), I suspect the number 
pings you will receive will increase. And I guess the same will happen with 
yardstick, that's why I don't think this is a SFC <--> Functest issue but a 
more general ways of working issue.

Regards,
Manuel



On Fri, 2018-02-02 at 14:24 +0000, 
cedric.olliv...@orange.com<mailto:cedric.olliv...@orange.com> wrote:
Hello Manuel,

In case of Functest, we are simply improving our code that why we are removing 
uncovered obsolete (and unused internally) functions.

As we take care of pylint output and coverage, we should stop maintaining 
modules/functions which are out of our standards and unused by Functest.
https://git.opnfv.org/functest/tree/tox.ini

It’s well known that we are getting rid of openstack utils to leverage on snaps 
and we are removing unused functest utils
We have worked on SDNVPN about that point before and we should apply the same 
rules for SFC.
I don’t understand why Functest should host functions needed by SFC.

In fact this issue is on the client side (SFC) which still uses obsolete 
Functest functions and your email seems asking to block innovation.
I don’t think it’s an issue regarding APEX master or XCI for which this global 
improvement initiated from E is very helpful.

Be free to reuse this method in your tree as it could have belong to instead.

Cédric

De : 
opnfv-tech-discuss-boun...@lists.opnfv.org<mailto:opnfv-tech-discuss-boun...@lists.opnfv.org>
 [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] De la part de Manuel Buil
Envoyé : vendredi 2 février 2018 13:00
À : OPNFV-TECH-DISCUSS OPNFV
Objet : [opnfv-tech-discuss] Could ways of working with testing frameworks be 
improved when working with master?

Hi,

As you know, the SFC project is currently testing openstack master, that means 
we need to use functest master in order to consume the lastest patches in the 
projects we use. Unfortunately, in the last weeks, we wasted quite some time 
trying to understand why suddenly the tests were not able to run and the cause 
was a change in the master branch of functest. For example, we are using the 
function "timethis" from functest_utils, which was removed by this patch: 
https://github.com/opnfv/functest/commit/c6092cb676363d89f366dc9a416ba6c53eeea33f#diff-b5f06ecfb223c80624f432ef33cf1fdd
 and suddenly our tests are not working and we don't know whether there is an 
alternative.

Functest is the framework we use for our tests and ideally, we (SFC project) 
would like to get some heads up before that change is done, so that we are 
warned and we don't have to waste time investigating what changed. I guess the 
same could be applied to other core testing frameworks like Yardstick. However, 
this is complicated and I am not sure if there is a good solution to achieve 
that level of communication without impacting the efficiency of 
funcatest/yardstick/... development. I have some ideas:

A) Functest/Yarstick leave the old functionality for a week adding a log saying 
"This is going to be deprecated, please check this patch: xxxx"

B) Add gates in functest/yardstick projects which run tests of their customer 
projects (as in, SFC is a customer of functest). This way, projects could be 
warned on time

C) Do nothing. Sorry, this is the consequence of consuming functest/yardstick 
master

D) ??

From what I heard in the TSC, Apex is going to join the XCI philosophy and 
allow working with the tip of master, so it seems to me that more functest and 
potentially yardstick users are going to hit this problem, that's why I believe 
it could be a good time to discuss possible solutions.

Please don't get me wrong, I am not trying to blame functest (or yardstick), I 
just want to share the problems we are having with the current ways of working 
and try to find a solution :)

Thanks,
Manuel

_________________________________________________________________________________________________________________________



Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc

pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler

a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,

Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.



This message and its attachments may contain confidential or privileged 
information that may be protected by law;

they should not be distributed, used or copied without authorisation.

If you have received this email in error, please notify the sender and delete 
this message and its attachments.

As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.

Thank you.

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations 
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce 
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages 
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou 
falsifie. Merci.

This message and its attachments may contain confidential or privileged 
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete 
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been 
modified, changed or falsified.
Thank you.

_______________________________________________
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss

Reply via email to