Hi Georg,

I think your proposal could be a good solution!! Having an API which
guarantees us some stability will surely solve the problems I
described. I am not sure if this will be too much effort for the
testing frameworks though and how the API model could be developed.

Regards,
Manuel

On Fri, 2018-02-02 at 14:42 +0000, Georg Kunz wrote:
> 
> 
> Hi all,
>  
> > > In the context of this discussion I am wondering if one of the
official goals of Functest is to provide common functionality to
support developers in OPNFV to write functional
> > >  tests? That was my understanding, but I might be wrong. In any case,
the Functest team along with the broader community should have a
clear common understanding of this.
>  
> > > Assuming this is the case, then the functionality which Functest
provides constitutes an API. Typically, APIs exhibit some kind of
stability along with some kind of deprecation
> >  process. It would in this case be beneficial for the broader
community if such processes are defined and known.
>  
> > > If Functest is not considered a framework providing common
functionality, well, then it is indeed a problem of SFC. However, I
do not consider this a valuable approach because
> > > >  it might just result in every project creating similar code for the
same tasks, e.g., measuring execution times as in Manuel’s case.
Moreover, Brian’s statement that he is trying to avoid using Functest
functionality in his projects doesn’t really feel right
>  from a community perspective…
>  
> Just my two cents as an observer.
>  
> Best regards
> Georg
>  
>  
> 
> 
> 
> > > From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-d
iscuss-boun...@lists.opnfv.org]
> On Behalf Of cedric.olliv...@orange.com
> 
> Sent: Friday, February 02, 2018 3:25 PM
> 
> > > To: Manuel Buil <mb...@suse.com>; OPNFV-TECH-DISCUSS OPNFV <opnfv-tec
h-disc...@lists.opnfv.org>
> 
> > Subject: Re: [opnfv-tech-discuss] Could ways of working with testing
frameworks be improved when working with master?
> 
> 
> 
> 
>  
> Hello Manuel,
>  
> > In case of Functest, we are simply improving our code that why we are
removing uncovered obsolete (and unused internally) functions.
>  
> > As we take care of pylint output and coverage, we should stop
maintaining modules/functions which are out of our standards and
> unused by Functest.
> https://git.opnfv.org/functest/tree/tox.ini
>  
> > It’s well known that we are getting rid of openstack utils to
leverage on snaps and we are removing unused functest utils
> > We have worked on SDNVPN about that point before and we should apply
the same rules for SFC.
> I don’t understand why Functest should host functions needed by SFC.
> 
>  
> > > In fact this issue is on the client side (SFC) which still uses
obsolete Functest functions and your email seems asking to block
innovation.
> > I don’t think it’s an issue regarding APEX master or XCI for which
this global improvement initiated from E is very helpful.
>  
> > Be free to reuse this method in your tree as it could have belong to
instead.
>  
> Cédric
>  
> 
> 
> De :
> > opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss
-boun...@lists.opnfv.org]
> De la part de Manuel Buil
> 
> Envoyé : vendredi 2 février 2018 13:00
> 
> À : OPNFV-TECH-DISCUSS OPNFV
> 
> > Objet : [opnfv-tech-discuss] Could ways of working with testing
frameworks be improved when working with master?
> 
> 
> 
> 
>  
> 
> Hi,
> 
> 
> 
>  
> 
> 
> 
> > > > As you know, the SFC project is currently testing openstack master,
that means we need to use functest master in order to consume the
lastest patches in the projects we use. Unfortunately, in the last
weeks, we wasted quite
> > > > > >  some time trying to understand why suddenly the tests were not able
to run and the cause was a change in the master branch of functest.
For example, we are using the function "timethis" from
functest_utils, which was removed by this patch: https://github.com/o
pnfv/functest/commit/c6092cb676363d89f366dc9a416ba6c53eeea33f#diff-
b5f06ecfb223c80624f432ef33cf1fdd and
> >  suddenly our tests are not working and we don't know whether there
is an alternative.
> 
> 
> 
>  
> 
> 
> 
> > > > Functest is the framework we use for our tests and ideally, we (SFC
project) would like to get some heads up before that change is done,
so that we are warned and we don't have to waste time investigating
what changed. I
> > > > >  guess the same could be applied to other core testing frameworks
like Yardstick. However, this is complicated and I am not sure if
there is a good solution to achieve that level of communication
without impacting the efficiency of funcatest/yardstick/...
development.
>  I have some ideas:
> 
> 
> 
>  
> 
> 
> 
> > > A) Functest/Yarstick leave the old functionality for a week adding a
log saying "This is going to be deprecated, please check this patch:
xxxx"
> 
> 
> 
>  
> 
> 
> 
> > > B) Add gates in functest/yardstick projects which run tests of their
customer projects (as in, SFC is a customer of functest). This way,
projects could be warned on time
> 
> 
> 
>  
> 
> 
> 
> > C) Do nothing. Sorry, this is the consequence of consuming
functest/yardstick master
> 
> 
> 
>  
> 
> 
> 
> D) ??
> 
> 
> 
>  
> 
> 
> 
> > > > From what I heard in the TSC, Apex is going to join the XCI
philosophy and allow working with the tip of master, so it seems to
me that more functest and potentially yardstick users are going to
hit this problem, that's
>  why I believe it could be a good time to discuss possible solutions.
> 
> 
> 
>  
> 
> 
> 
> > > Please don't get me wrong, I am not trying to blame functest (or
yardstick), I just want to share the problems we are having with the
current ways of working and try to find a solution :) 
> 
> 
> 
>  
> 
> 
> 
> Thanks,
> 
> 
> 
> Manuel
> 
> 
> > _____________________________________________________________________
____________________________________________________
>  
> > Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
> > pas etre diffuses, exploites ou copies sans autorisation. Si vous
avez recu ce message par erreur, veuillez le signaler
> > a l'expediteur et le detruire ainsi que les pieces jointes. Les
messages electroniques etant susceptibles d'alteration,
> > Orange decline toute responsabilite si ce message a ete altere,
deforme ou falsifie. Merci.
>  
> > This message and its attachments may contain confidential or
privileged information that may be protected by law;
> they should not be distributed, used or copied without authorisation.
> > If you have received this email in error, please notify the sender
and delete this message and its attachments.
> > As emails may be altered, Orange is not liable for messages that have
been modified, changed or falsified.
> Thank you.
> 
> 
> 
> 
> 
> 
_______________________________________________
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss

Reply via email to