Hi Morgan I agree with you that many of these challenges are directly related to “what we are producing”. I think the Dovetail project is bringing some of the issues out to the surface. During the Summit, we also shared the analysis we went through for all projects in Danube to evaluate if some of the test cases could be incorporated into Dovetail, and the consensus of the participants was the scope we proposed. We think it is a sound first step for CVP.
Specifically to two ideas you brought up: vIMS would be a great addition but unfortunately it was not maintained in Danube, as you noted. We could not include a test that does not run in the release that the cvp program is based on. I hope we can work together to have vIMS in E release. In terms of “SLA” testing with Yardstick, performance testing is out of scope for now. We are just beginning to discuss/learn how to properly define and measure performance metrics. The discussions during the plugfest and the Summit had been very useful to get started. Yardstick HA testing is the only test area included in scope for now. Regards Wenjing From: opnfv-tsc-boun...@lists.opnfv.org [mailto:opnfv-tsc-boun...@lists.opnfv.org] On Behalf Of morgan.richo...@orange.com Sent: Tuesday, June 27, 2017 2:32 AM To: Wenjing Chu <chu.wenj...@gmail.com>; Tim Irnich <tim.irn...@ericsson.com> Cc: TSC OPNFV <opnfv-...@lists.opnfv.org>; TECH-DISCUSS OPNFV <opnfv-tech-discuss@lists.opnfv.org> Subject: Re: [opnfv-tsc] [opnfv-tech-discuss] [dovetail] TSC and DoveTail meeting to discuss scope and needs for CVP testing Hi, my view is that the difficulty we have to converge to a clear consensus is directly linked to what we are producing I fully agree with Fatih's comment on the mail https://lists.opnfv.org/pipermail/opnfv-tech-discuss/2017-June/016799.html Second release had probably the best quality in term of release, since the second release we deal with a wild rush forward "more everything" and releases are somehow snapshots of current scenarios more or less (usely less) tested - that is one of the reasons why the testing group is proposing a new way for resiliency/stress testing - discussion planned during the TSC meeting today - So it is not surprising to see several interpretations of what certification should be. If we were able to say clearly "an OPNFV release is X, Y and Z" it would be much easier. But we are dealing with a composite object with lots of possible combinations, features (even mature ones) have installer/scenario constraints. The second difficult point I see then is that due to the complexity of the combinations, we reduced the initial scope, which is fine, but then the question of the delta compared to OpenStack can be raised. Adding features that are supported only by a subset of installers/scenarios is a way to differentiate but as it is only a subset, does it make sense to consider it for an OPNFV certification? CVP WG and Dovetail projects have been working hard for a long time and the last proposal is surely the best we can have regarding the context if we consider OPNFV as a product but is it a product? If we consider OPNFV as a framework and want to focus on NFVI/VIM, running yardstick and be sure that the NFVI reached all the defined SLAs makes sense for me. In Brahmaputra we were able to successfully test the deployment of a vIMS on several scenarios/installers (more than 1000 CI run done), this test case was complete to test VIM/NFVI and went far beyond a check of the OpenStack API/Interface. Since Colorado due to the problem mentioned earlier it is unfortunately not so stable and it was poorly tested in Danube (wait for a weekly job we were not able to reach / test/scenario promotion, trust indicator,...) /Morgan On 27/06/2017 01:46, Wenjing Chu wrote: I updated the document wiki page with the scope summarized in this email and the latest test spec documents: https://wiki.opnfv.org/display/dovetail/Dovetail+Documentation+for+Review. Is there any other feedback from tsc members? Tapio, I'll be on the tsc call tomorrow to answer any questions about the proposal. Can we have some time on the agenda? Thanks. Wenjing On Fri, Jun 23, 2017 at 2:50 AM, Tim Irnich <tim.irn...@ericsson.com<mailto:tim.irn...@ericsson.com>> wrote: Hi Dave, all, Sorry for misunderstanding your point. In that case, is there any other feedback from other TSC members on the proposal? Tapio & Ray, I think we should reserve some time in next week's TSC to go over the suggested test scope (both mandatory and optional parts) for Danube compliance testing once more so that the Dovetail team can be confident about focusing on the right things. Regards, Tim -----Original Message----- From: Dave Neary [mailto:dne...@redhat.com<mailto:dne...@redhat.com>] Sent: Wednesday, June 21, 2017 01:53 To: Tim Irnich <tim.irn...@ericsson.com<mailto:tim.irn...@ericsson.com>>; Wenjing Chu <wenjing....@huawei.com<mailto:wenjing....@huawei.com>>; Christopher Price <christopher.pr...@ericsson.com<mailto:christopher.pr...@ericsson.com>>; Tianhongbo <hongbo.tianhon...@huawei.com<mailto:hongbo.tianhon...@huawei.com>>; Tallgren, Tapio <tapio.tallg...@nsn.com<mailto:tapio.tallg...@nsn.com>>; Georg Kunz <georg.k...@ericsson.com<mailto:georg.k...@ericsson.com>> Cc: TSC OPNFV <opnfv-...@lists.opnfv.org<mailto:opnfv-...@lists.opnfv.org>>; TECH-DISCUSS OPNFV <opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org>> Subject: Re: [opnfv-tech-discuss] [opnfv-tsc] [dovetail] TSC and DoveTail meeting to discuss scope and needs for CVP testing Hi Tim, On 06/20/2017 09:02 PM, Tim Irnich wrote: >> I would like to see us document some of the NFV related requirements >> which are common across all RFCs from telcos, and which are available >> in all viable VIM products. > > This is exactly the intention of the proposal, under the side > constraint of drawing from already existing tests. The question to the > TSC was if this is enough for an initial release. I think your answer is no. On the contrary - the initial release scope is fine, my comment was on the "future plans" piece. Thanks, Dave. -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182<tel:%2B1-978-399-2182> / Cell: +1-978-799-3338<tel:%2B1-978-799-3338> _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org<mailto:opnfv-tech-discuss@lists.opnfv.org> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss -- Morgan Richomme Orange/ IMT/ OLN/ CNC/ NCA/ SINA Network architect for innovative services Future of the Network community member Open source Orange community manager tel. +33 (0) 296 072 106 mob. +33 (0) 637 753 326 morgan.richo...@orange.com<mailto:morgan.richo...@orange.com> _________________________________________________________________________________________________________________________ Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci. This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified. Thank you.
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
