Team, We had a productive meeting today with the stakeholders and developed a plan to resolve this request. Meeting notes and an action plan may be found here <https://wiki.onap.org/pages/viewpage.action?pageId=71827884>. We will have a followup meeting to discuss progress on Oct 11. Let me know if you have any questions.
David On Wed, Sep 18, 2019 at 10:44 AM David McBride <[email protected]> wrote: > +Arpit Joshipura <[email protected]> > > I'm looking into the issue and will update this thread as soon as I have > additional information. Unfortunately, the two people that know the most > about this issue are currently unavailable. I'll keep you posted. > > David > > On Wed, Sep 18, 2019 at 12:44 AM <[email protected]> wrote: > >> Can this be moved forward? And if there is a blocking point, can we know >> where it is? >> >> Kind regards >> >> Vincent >> >> >> >> *From:* [email protected] [mailto:[email protected]] *On >> Behalf Of *Sylvain Desbureaux via Lists.Onap.Org >> *Sent:* mercredi 18 septembre 2019 09:02 >> *To:* [email protected]; [email protected]; David >> McBride; Andrew Grimberg >> *Cc:* [email protected]; DEBEAU Eric TGI/OLN; >> [email protected]; [email protected]; >> [email protected]; [email protected]; RICHOMME Morgan >> TGI/OLN; DANNO Vincent TGI/OLN >> *Subject:* Re: [onap-tsc] [Fwd: Re: [ONAP] Gating evolution] >> >> >> >> Hello, >> >> It's been: >> >> * 3 months since we asked for KaaS resources during DDF in Stockholm (and >> the approval from TSC and PTLs) >> >> * 2 months since we (re)asked LFN for the KaaS resources >> >> * 3 weeks since first technical discussion with Andrew >> >> * a week and a half since last email from LFN on this topic (which was >> addressing one of the many questions we have and not the most important >> one: when do we have the resources). >> >> >> >> We already missed El Alto (and Dublin…) priority on testing, I don’t want >> to miss also Frankfurt one. >> >> >> >> So, as it seems that you have issues to provide the solution you proposed >> 3 weeks ago, can we go back to the initial request (credentials to a KaaS >> provider) so we can move forward ASAP? >> >> >> >> For your information, CNCF, your sister foundation, is able to give >> credentials for hosted project (Kubernetes, Network Service Mesh (NSM), >> ...) on a lot of different platforms (NSM is deploying on GKE, AWS, Azure, >> Packet, …) so where there's a will there's a way. >> >> >> >> Hoping that have credentials very soon, >> >> >> >> >> >> --- >> >> Sylvain Desbureaux >> >> >> >> *De : *RICHOMME Morgan TGI/OLN <[email protected]> >> *Date : *mardi 20 août 2019 à 10:44 >> *À : *"[email protected]" <[email protected]>, " >> [email protected]" <[email protected]> >> *Cc : *"[email protected]" <[email protected]>, DEBEAU Eric >> TGI/OLN <[email protected]>, DESBUREAUX Sylvain TGI/OLN < >> [email protected]>, "[email protected]" < >> [email protected]>, "[email protected]" <[email protected]>, " >> [email protected]" <[email protected]> >> *Objet : *[Fwd: Re: [ONAP] Gating evolution] >> >> >> >> Hi Kenny >> >> >> >> as discussed yesterday, you will find hereafter the mail thread on the >> ONAP gating evolution sent on the 20th of June. >> >> >> >> this thread was initiated before the DDF in June and presented during PTL >> meeting (8th of July: https://wiki.onap.org/display/DW/PTL+2019-07-08) >> for official approval. >> >> A slide deck had been shared with PTLs and TSC: >> https://wiki.onap.org/download/attachments/64010469/OOM_CD_Gating_ptl.pdf?version=1&modificationDate=1562598568000&api=v2 >> >> A first estimation of the APEX cost was mentioned in the mail below and >> discussed during ad hoc meetings. >> >> The last mail on the mailing list dealing with the topic can be found >> here >> https://lists.onap.org/g/onap-discuss/message/17927?p=,,,20,0,0,0::Created,,Gating+evolution,20,2,0,32393410 >> >> >> >> It corresponds to a real and immediate community need. >> >> >> >> Regards >> >> >> >> Morgan >> >> >> >> ND: In parallel there are general discussions on mid term evolution of >> the CI/CD at LFN level (reporting done last week). >> >> I already mentioned during TAC meeting that I am a bit puzzled on the way >> LF manages the hardware resources (I asked several times for the list of LF >> HW community resources accross the different projects but never got an >> answer) and the CI/CD. >> >> If we consider OPNFV CI for instance, we can see that lots of the runs >> today correspond to projects that are sometimes finished since several >> months (old installers still running) or even years, run on old versions >> (almost no run on iruya...except functest), very high error rate on some >> jobs meaning that the jobs are clearly not maintained. >> >> There is probably a big waste of resources (and by consequence money). >> >> It shows also the difficulty and the challenges of a centralized model. >> >> Once community contributors stop their activity, they do not clean their >> job as the CI is out of their scope, the jobs remain because the >> centralized organization in charged of the CI has no idea that it is >> finished. The minutes of CI do not reflect the reality of the community. >> (same syndrome for the repos, some should have been archived since a long >> time to reflect the real activity of the community) >> >> >> >> See >> https://opnfv.biterg.io/app/kibana#/dashboard/00862e30-bbf7-11e8-8aac-ef7fd4d8cbad?_g=()&_a=(description:'Jenkins%20Jobs%20panel%20by%20Bitergia',filters:!(),fullScreenMode:!f,options:(darkTheme:!f,useMargins:!t),panels:!((gridData:(h:4,i:'3',w:5,x:7,y:0),id:JenkinsResults,panelIndex:'3',title:Results,type:visualization,version:'6.1.0-3'),(gridData:(h:2,i:'8',w:7,x:0,y:0),id:jenkins_main_numbers,panelIndex:'8',title:Summary,type:visualization,version:'6.1.0-3'),(gridData:(h:7,i:'11',w:12,x:0,y:7),id:a44d0f70-bbf7-11e8-8aac-ef7fd4d8cbad,panelIndex:'11',title:Jobs,type:visualization,version:'6.1.0-1'),(gridData:(h:2,i:'13',w:7,x:0,y:2),id:e0236790-bc12-11e8-8aac-ef7fd4d8cbad,panelIndex:'13',title:Jobs,type:visualization,version:'6.1.0-1'),(gridData:(h:3,i:'15',w:5,x:7,y:4),id:bedd3e00-bcab-11e8-8aac-ef7fd4d8cbad,panelIndex:'15',title:'Success%2FFailures%20in%20percentage',type:visualization,version:'6.1.0-1'),(gridData:(h:3,i:'16',w:7,x:0,y:4),id:f5a4cdb0-bcae-11e8-8aac-ef7fd4d8cbad,panelIndex:'16',title:'Duration%20Trend',type:visualization,version:'6.1.0-1')),query:(language:lucene,query:(query_string:(analyze_wildcard:!t,default_field:'*',query:'*'))),timeRestore:!f,title:'Jenkins%20Jobs',uiState:(P-10:(title:Buids),P-11:(spy:(mode:(fill:!f,name:!n)),title:'Export%20Builds',vis:(params:(config:(searchKeyword:''),sort:(columnIndex:!n,direction:!n)))),P-12:(spy:(mode:(fill:!f,name:!n)),vis:(legendOpen:!f)),P-13:(vis:(legendOpen:!f)),P-3:(title:Results),P-4:(title:Nodes,vis:(params:(config:(searchKeyword:''),sort:(columnIndex:!n,direction:!n)))),P-5:(title:'Builds%20over%20time',vis:(legendOpen:!f)),P-8:(spy:(mode:(fill:!f,name:!n)),title:Summary),P-9:(spy:(mode:(fill:!f,name:!n)),title:'Active%20nodes%20over%20time',vis:(legendOpen:!f))),viewMode:view >> ) >> >> >> >> >> >> >> >> >> >> -------- Message transféré -------- >> >> *De*: DESBUREAUX Sylvain TGI/OLN <[email protected] >> <DESBUREAUX%20Sylvain%20TGI/oln%20%[email protected]%3e>> >> >> *À*: [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>> >> >> *Cc*: DEBEAU Eric TGI/OLN <[email protected] >> <DEBEAU%20Eric%20TGI/oln%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, >> DANNO Vincent TGI/OLN <[email protected] >> <DANNO%20Vincent%20TGI/oln%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e>>, [email protected] >> <[email protected] <%[email protected]%22%20%[email protected]%3e>>, RICHOMME >> Morgan TGI/OLN <[email protected] >> <RICHOMME%20Morgan%20TGI/oln%20%[email protected]%3e>>, >> [email protected] <[email protected] >> <%[email protected]%22%20%[email protected]%3e> >> > >> >> *Objet*: Re: [ONAP] Gating evolution >> >> *Date*: Thu, 04 Jul 2019 09:15:24 +0200 >> >> >> >> Hello, >> >> Do you have any news on this topic? >> >> El Alto release is due for September 30th and first tests are expected >> end of July. >> >> If we want to (try to) start automated gating for first commits, we need >> credentials as soon as possible (before July 9th) and it’s not _*at all*_ >> guaranteed. >> >> >> >> As summer is the holiday season, if we don’t have soon access to a lab, >> nothing will be done before mid/end august so expect first tests >> beginning/mid September. >> >> >> >> So this will be done only for the last session of test at best, and there >> would be an high risk to delay El Alto release (as we have seen in >> Casablanca and Dublin release). >> >> >> >> El Alto release is supposed to be focused on Test Automation & CI/CD >> pipeline as stated in the wiki but I’ve not seen anything on this since >> Stockholm DDF… >> >> >> >> Regards, >> >> >> >> --- >> >> Sylvain Desbureaux >> >> >> >> *De : *RICHOMME Morgan TGI/OLN <[email protected]> >> *Date : *jeudi 20 juin 2019 à 18:43 >> *À : *"[email protected]" <[email protected]>, >> "[email protected]" <[email protected]>, " >> [email protected]" <[email protected]>, " >> [email protected]" <[email protected]>, " >> [email protected]" <[email protected]> >> *Cc : *DEBEAU Eric TGI/OLN <[email protected]>, " >> [email protected]" <[email protected]>, " >> [email protected]" <[email protected]>, " >> [email protected]" <[email protected]>, DESBUREAUX Sylvain TGI/OLN >> <[email protected]>, "[email protected]" < >> [email protected]>, DANNO Vincent TGI/OLN < >> [email protected]>, "[email protected]" <[email protected]>, " >> [email protected]" <[email protected]> >> *Objet : *[ONAP] Gating evolution >> >> >> >> Hi >> >> >> >> as discussed during LFN DDF in Stockholm, we come back to you to discuss >> the possibility to get additional resources for community activities. >> >> As you know, the gating feature now allows to detect regression before >> merge on the installer repository. >> >> It has been introduced for Dublin release and is highly appreciated by >> the community. >> >> >> >> This light Openstack-zuul like feature requires some infrastructure in >> order to deploy a full fresh ONAP and test it within a CI chain. >> >> So far all the resources used to provide this service to the community >> are fully provided by Orange. >> >> In fact the on-demand deployments are done in Orange lab, operated by >> Orange team. >> >> >> >> In order to get geographical redundancy and share the load, it would >> fully make sense to add additional resources. >> >> Orange team is OK to participate to the setup/monitoring of the gating >> chain on these additional resources. >> >> >> >> Several options are possible: another community member hosting platforms, >> Cloud providers... >> >> >> >> We calculated some time ago that the dimensioning for the installer >> gating lead to 9 platforms in parallel at peak (for Dublin). >> >> Today we manage the gating with 1 or 2 platforms, as a consequence we >> added a queuing mechanism. >> >> Assuming that a full gating processing lasts about 1h30, if your patch is >> queued #5 and you have only 1 platform available, you have to wait 6x1h30 = >> 9h before getting the feedback on your patch. >> >> With 9 platforms, the max would have been 1h30. >> >> >> >> This dimensioning deals only with the gating of 1 ONAP sub-project: the >> installer project OOM. >> >> During DDF and in various presentations before, it was indicated that >> gating on key components would be also a good improvement. the SO component >> would be candidate for such gating. >> >> The gating of the SO would be relatively similar to the OOM gating. It >> requires some evolution on the docker generation (currently mainly managed >> by LF IT team) especiallyon docker generation and versioning. >> >> It does not look very complex but some work is needed. >> >> >> >> Of course if we add component gating, the dimensioning will change >> dramatically. >> >> As we can start we only 1 subcomponent, it is possible to scale as we >> grow. But in any case 2 // platform will be too light. >> >> >> >> In Orange for the gating we are deploying a kubernetes solution on top of >> our DC (Openstack). A full ONAP means 18 Kubernetes nodes >> >> - 3 controller nodes: each node= 4 cores / 4 Go RAM / 40 Go Disk >> >> - 3 etc nodes: each node= 4 cores / 4Go RAM / 40 Go Disk >> >> - 12 compute nodes: each node= 16 cores / 32 Go RAM / 40 Go Disk >> >> total for 1 platform = 216 cores / 408 Go RAM / 720 Go Disk >> >> >> >> we may notice that is is still very big and probably an obstacle for >> adoption...but it is another debate... >> >> >> >> Let's do the hypothesis we setup 2 new platforms for gating in public >> cloud >> >> the first evaluations of the costs are (just using quickly online >> calculator of the different providers based on a 100% usage per month, >> which is probably not the case; need to contact them directly for more >> accuracy) >> >> - Google GKE: ~10K$ ( >> https://cloud.google.com/products/calculator/#tab=container) >> >> - Amazon EKS ~ 10K$ / month (https://aws.amazon.com/eks/pricing/?nc1=h_ls >> ) >> >> - Azure: ~ 18K$ / month ( >> https://azure.microsoft.com/en-gb/pricing/calculator/) >> >> - VMWare: ~10K€ (https://cloud.vmware.com/vmware-cloud-pks/pricing) >> >> - DigitalOcean: 8 K$/ month (https://www.digitalocean.com/pricing/) >> >> - OVH: ~ 15K$ (https://www.ovh.com/fr/public-cloud/prices/) >> >> - IBM: did not find the pricing page >> >> >> >> In a first stage, it would maybe make sense to ask community members >> (Microsoft, VMWare, IBM...) offering Kubernetes as a service offers if it >> would be possible to have credits for such community work. >> >> >> >> If TSC is OK and if we got the cloud credentials, Orange team would be >> pleased to contribute with integration/CI/CD team setting up CI chains for >> gating on these new resources. >> >> >> >> Regards >> >> >> >> Morgan >> >> _________________________________________________________________________________________________________________________ >> >> >> >> Ce message et ses pieces jointes peuvent contenir des informations >> confidentielles ou privilegiees et ne doivent donc >> >> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu >> ce message par erreur, veuillez le signaler >> >> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages >> electroniques etant susceptibles d'alteration, >> >> Orange decline toute responsabilite si ce message a ete altere, deforme ou >> falsifie. Merci. >> >> >> >> This message and its attachments may contain confidential or privileged >> information that may be protected by law; >> >> they should not be distributed, used or copied without authorisation. >> >> If you have received this email in error, please notify the sender and >> delete this message and its attachments. >> >> As emails may be altered, Orange is not liable for messages that have been >> modified, changed or falsified. >> >> Thank you. >> >> >> >> _________________________________________________________________________________________________________________________ >> >> Ce message et ses pieces jointes peuvent contenir des informations >> confidentielles ou privilegiees et ne doivent donc >> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu >> ce message par erreur, veuillez le signaler >> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages >> electroniques etant susceptibles d'alteration, >> Orange decline toute responsabilite si ce message a ete altere, deforme ou >> falsifie. Merci. >> >> This message and its attachments may contain confidential or privileged >> information that may be protected by law; >> they should not be distributed, used or copied without authorisation. >> If you have received this email in error, please notify the sender and >> delete this message and its attachments. >> As emails may be altered, Orange is not liable for messages that have been >> modified, changed or falsified. >> Thank you. >> >> > > -- > *David McBride* > Release Manager > Linux Foundation Networking (LFN) > Mobile: +1.805.276.8018 > Email/Google Talk: [email protected] > IRC: dmcbride > -- *David McBride* Release Manager Linux Foundation Networking (LFN) Mobile: +1.805.276.8018 Email/Google Talk: [email protected] IRC: dmcbride -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#5443): https://lists.onap.org/g/onap-tsc/message/5443 Mute This Topic: https://lists.onap.org/mt/34185316/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-tsc/leave/2743226/1412191262/xyzzy [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
