While it might be the overall concern on ONAP, we are discussing implementation of K8S Plugin ☺.
So if we can address the concern here, it will certainly contribute to the future discussion of overall concern ☺ Bin From: Addepalli, Srinivasa R [mailto:srinivasa.r.addepa...@intel.com] Sent: Thursday, June 14, 2018 8:38 AM To: HU, BIN <bh5...@att.com>; HEKMAT, ARASH <arash.hek...@amdocs.com>; Yang, Bin (Wind River) <bin.y...@windriver.com>; Morales, Victor <victor.mora...@intel.com>; onap-discuss@lists.onap.org Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com>; Huang, Yun (Wind River) <yun.hu...@windriver.com> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Hi Bin, I think you are raising a concern across ONAP. Today, following languages are used in ONAP (Even not considering K8S plugin) – From dec 17th presentation by Jason (Software Architecture 11 December) - Java - Python - Java script - C - Go - Perl - Erlang - NodeJS - Clojure - Shell - Lua You are suggesting that ONAP should recommend few languages so that we have skillset that can support ONAP in future. So far, this discussion did not come up except in this list ☺ It would be good to raise at ONAP TSC/Architecture level? Thanks Srini From: HU, BIN [mailto:bh5...@att.com] Sent: Thursday, June 14, 2018 7:39 AM To: HEKMAT, ARASH <arash.hek...@amdocs.com<mailto:arash.hek...@amdocs.com>>; Addepalli, Srinivasa R <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda I think all of the arguments now focus on the technology side, its merit and how to implement it. From user experience perspective, my interest is more related to how I can get continuous community support to maintain and evolve it in the future. When I invest in using something, current technology and implementation is certainly important (i.e. CapEx). But more importantly, after using it in operation, getting the continuous support from community is vital to the success of operation (i.e. OpEx). Of course, we can expect people to learn, but that is the best wish. Currently, I don’t see a convincing plan of growing the support of a particular language (Go) except the best wish. Thus I have the concern of sustainable support (e.g. OpEx) from user perspective. Hope my point can be really understood, which has nothing to do technology itself (e.g. micro-service, performance etc.) Thanks Bin From: HEKMAT, ARASH Sent: Thursday, June 14, 2018 6:57 AM To: Addepalli, Srinivasa R <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; HU, BIN <bh5...@att.com<mailto:bh5...@att.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda In my view, the whole idea of micro-services is isolation of technologies and communications only through well-defined APIs. So the arguments both for and against a specific language for a micro-service based on the language having been used or not used in other micro-services are contrary to the idea of micro-services design. So I believe: 1. As long as a micro-service is self-contained, both in code and at run time, and has well-defined APIs for all communications with other parts of the system then the choice of technology for the micro-service should only be based on what best suits the requirements of the micro-service. 2. To me ultimately the decision on what technology to use for a micro-service should be with the PTL and the team that commits to implementing the micro-service (committed implementers can vote). Regards, Arash From: Addepalli, Srinivasa R [mailto:srinivasa.r.addepa...@intel.com] Sent: Wednesday, June 13, 2018 6:07 PM To: HU, BIN <bh5...@att.com<mailto:bh5...@att.com>>; Arash Hekmat <arash.hek...@amdocs.com<mailto:arash.hek...@amdocs.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Hi Bin, I think you answered questions yourself ☺. It is easy to learn, secure language, performant and many are eager to learn. There are some good number of people with Golang experience (thanks to some of the work that was done in Beijing) already in ONAP community. Also, - we see the need for using Prometheus (or similar) for event/statistics aggregation micro service to satisfy the MVP of Edge-automation use case. Since Prometheus is developed in golang, any ancillary packages/glues are also expected to be developed in golang. - ISTIO service mesh technology is being seriously looked at to replace existing MSB and simplify security. If this is adopted, any enhancements related to service mesh would normally happen in golang as ISTIO is developed in golang. With above two, I have a confidence that we will have more developers in golang. Thanks Srini From: HU, BIN [mailto:bh5...@att.com] Sent: Wednesday, June 13, 2018 12:26 PM To: Addepalli, Srinivasa R <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; HEKMAT, ARASH <arash.hek...@amdocs.com<mailto:arash.hek...@amdocs.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Hi Srini and Arash, Thank you for the great information. I am truly with you, and there is no doubt that K8S can be implemented with Go language, and I do see the merit of Go language that will provide great value in implementation. The concern here is not related to technology of Go language, as I mentioned earlier. Please understand that I do believe that Go language is easy to learn, and savvy developers all eager to learn new technologies. GoLang client is also well maintained by Kubernetes community as Victor mentioned in a different email. The situation is that K8S Plugin will be implemented and maintained in ONAP community. So for the long-term sustainability, a reasonable size of developer pool in ONAP community is needed to continuously maintain our own K8S Plugin code base in ONAP. I just want to know: do you have a plan to grow a reasonable size of developer pool in ONAP community so that K8S Plugin implementation with Go can be continuously maintained and evolved in ONAP? Thanks Bin From: Addepalli, Srinivasa R [mailto:srinivasa.r.addepa...@intel.com] Sent: Wednesday, June 13, 2018 7:36 AM To: HEKMAT, ARASH <arash.hek...@amdocs.com<mailto:arash.hek...@amdocs.com>>; HU, BIN <bh5...@att.com<mailto:bh5...@att.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Yes Arash. K8S plugin has its own repo and will be set (currently one) micro service run as container. We don’t intend to call Python calls from Go programs. So, answer is Yes for all three points you have mentioned in your email. Hi Bin, I don’t mean to be pedantic here. Sorry if it appears so. 1. (Can be argumentative and may become philosophical discussion)It is very easy to learn go-lang. I don’t see it as big issue either maintaining it or even developing. Engineers who contributed to SMS, Distributed KV Store (done as part of R2) were new to go and could learn in matter of days/weeks. 2. Code size is very small – Hence easy to adopt in Edge locations (if there is a need in future). 3. Static type checking – Less error prone. 4. Performance of golang based system is higher. (Bring up is very fast and it is just 10-20% lower performance than C/C++) 5. K8S Client in go-lang always little bit ahead of other language bindings. Go-client is maintained as part of Kubernetes project. 6. GoCrypto has a way to talk to HSMs. If there is a need to secure keys, password, secrets, then the support is already available. Java Crypto has similar support, but python does not have it (yet). 7. Many micro service projects are developed in golang. It is quite mature. Many CNCF projects are developed in golang. https://hackernoon.com/5-reasons-why-we-switched-from-python-to-go-4414d5f42690<https://urldefense.proofpoint.com/v2/url?u=https-3A__hackernoon.com_5-2Dreasons-2Dwhy-2Dwe-2Dswitched-2Dfrom-2Dpython-2Dto-2Dgo-2D4414d5f42690&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6qPcDOqMgwf1K_r6YIIHhw&m=kKJpI7d4BvGzsfgElthwPCjo7-CR9Z7D0x-r3mXt0KE&s=TJX91uvzwLwPtQBNruqU6B98r2WHw7rjpEjc9diS2OU&e=> Since ONAP is micro-service based, language choice for a given micro-service should not matter. And you will find more and more Engineers with go skillset with increasing popularity Kubernetes and micro-service architecture Thanks Srini From: Arash Hekmat [mailto:arash.hek...@amdocs.com] Sent: Wednesday, June 13, 2018 7:04 AM To: HU, BIN <bh5...@att.com<mailto:bh5...@att.com>>; Yang, Bin (Wind River) <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; Addepalli, Srinivasa R <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; Morales, Victor <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua (Wind River) <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun (Wind River) <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: RE: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Hi MultiCloud Team, My 2 cents here. I believe if below conditions are true, the K8S plugin can be implemented in Go or any other language. But if they are not true, it should probably match MultiCould’s core and use Python, for all the maintainability and integration concerns already raised. The K8S plugin can use Go or any other language if: (1) It has its own separate git repo (2) It runs in its own separate docker container (3) It does not directly call or share MultiCould’s Python framework Arash From: onap-discuss-boun...@lists.onap.org<mailto:onap-discuss-boun...@lists.onap.org> [mailto:onap-discuss-boun...@lists.onap.org] On Behalf Of HU, BIN Sent: Wednesday, June 13, 2018 2:02 AM To: Yang, Bin <bin.y...@windriver.com<mailto:bin.y...@windriver.com>>; ADDEPALLI, SRINIVASA <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; MORALES RUVALCABA, VICTOR <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Cc: Zhang, Xiaohua <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>>; Huang, Yun <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>> Subject: Re: [onap-discuss] Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda If I understand correctly, there are 2 types of concerns regarding Go language (v.s. Python) here: - Implementation, integration and testing: e.g. CI/CD integration, reuse of shared services etc. - Long-term community support with the appropriate expertise in Go language. The 2nd one, i.e long-term community support is actually the biggest concern. For example, after K8S plugin is developed with Go language, but the original Go developer no longer works in ONAP, do we have a pool of Go developers in ONAP community that can comfortably take over the work of maintenance and further evolving K8S Plugin? Or will then-current developers have to re-develop K8S Plugin with Python or Java because most of ONAP developers are Python or Java developers? If there are very few Go developers in ONAP community, using Go language certainly brings huge risk of continuous support, maintenance and evolution in the future. My 2 cents Thanks Bin From: Yang, Bin [mailto:bin.y...@windriver.com] Sent: Tuesday, June 12, 2018 8:27 PM To: ADDEPALLI, SRINIVASA <srinivasa.r.addepa...@intel.com<mailto:srinivasa.r.addepa...@intel.com>>; MORALES RUVALCABA, VICTOR <victor.mora...@intel.com<mailto:victor.mora...@intel.com>>; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org>; HU, BIN <bh5...@att.com<mailto:bh5...@att.com>> Cc: Huang, Yun <yun.hu...@windriver.com<mailto:yun.hu...@windriver.com>>; HUANG, HAIBIN <haibin.hu...@intel.com<mailto:haibin.hu...@intel.com>>; Zhang, Xiaohua <xiaohua.zh...@windriver.com<mailto:xiaohua.zh...@windriver.com>> Subject: Building blocks for multicloud k8s plugin, RE: [coe] Team meeting Agenda Hi Victor, Srini and all, In addition to the agenda you listed, could you also add one more to touch the topic of building blocks (e.g. “language binding” with golang instead of python) for multicloud k8s plugin? I do understand that from the architecture perspective it is irrelevant w.r.t. how to implement the plugin service, hence up to developers/contributors’ choice. However, it is worth to think again for following reasons: 1, Leverage and Scale Aligning to existing practice of multicloud plugin implementation enables developers focus on the core value of the plugin, e.g. mediate ONAP orchestration to k8s cluster in case of multicloud k8s plugin. As part of ONAP projects it is required that the multicloud k8s plugin be integrated with CI/CD process and other shared services (e.g. AAF, centralized logging, etc.) Right now CI/CD support projects with python binding very well, and the logging AOP facility for Python is well supported, I don’t know if there is off-the-shelf utility for golang available yet. If multicloud k8s plugin align to the existing practice of openstack plugin and vio plugin, then you don’t have to invest further time/cost on those effort. 2, Facilitate more participation from current multicloud committers/contributors multicloud committers/contributors has been working on python already for a long time , and it is not surprising that many of them are willing to participate/contribute to this multicloud k8s plugin development. Assuming that Golang is not as widely adopted as Python so most of them is not that familiar with Golang compared to Python, it would lower the barrier for them to contribute if multicloud k8s plugin could adopt the same language and framework. 3, Decrease the maintenance cost from either community and the Service Providers who will productize ONAP By align to the same practice and share the same building blocks across multicloud plugins, one committer/contributor support/maintain one plugin could also be possible to support another one with low cost to learn/understand it. This will decrease the TCO which is always a pain of open source product compared to those commercialized close source product. 4, facilitate pairwise/integration testing for fast release cadence It might not concern those architect, but it is really a pain for developers who support the pairwise/integration testing. Fortunately, python make our life better since it is possible to fix the bug on runtime and test it immediately, which reduce the cycle of debugging. I can share you some feedback from several PTLs whom I had supported during Pairwise/integration test if you are interested to know. So it is my 2 cents, please let me know if you have other concerns/opinions and I am willing to learn them, thanks. Best Regards, Bin Yang, Solution Readiness Team, Wind River Direct +86,10,84777126 Mobile +86,13811391682 Fax +86,10,64398189 Skype: yangbincs993 From: onap-discuss-boun...@lists.onap.org<mailto:onap-discuss-boun...@lists.onap.org> [mailto:onap-discuss-boun...@lists.onap.org] On Behalf Of Yang, Bin Sent: Wednesday, June 13, 2018 9:21 AM To: ADDEPALLI, SRINIVASA; MORALES RUVALCABA, VICTOR; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: Re: [onap-discuss] [coe] Team meeting Agenda Hi Srini, and all, With regarding to the “each deployment instance needs to be uniquely identified”, what is the intention for this “uniquely identified”? I mean, who will care about/consume this unique ID? IMHO, from multicloud NBI consumers’ perspective, I don’t get the point to identify uniquely each deployment instance of multicloud plugin for following reasons: 1, every multicloud plugin could be deployed as a cluster/replicated set of “deployment instances” within a single ONAP instance, but they all should expose the NBI via the same endpoint. I assume every multicloud plugin are stateless service so that the consumer’s requests can be distributed via MSB or some other load balance mechanism . In that case multicloud consumers are not aware the “multiple deployment instances” of any multicloud plugin. 2, every multicloud plugin can also be deployed in the distributed way (e.g. deployed along with each edge ONAP instance). In this case the each multicloud deployment instance exposes unique NBI endpoint since the IP address of hosts (which are used to deploy the multicloud plugins) are different, so the consumers should be aware of which edge ONAP they are talking to. With either case the multicloud consumers are not necessarily identify uniquely the deployment instance. You may also wonder to know that how the consumers (SO, APPC,VFC) know what the multicloud plugin deployment instance’s NBI endpoint is in either cases above, here is my understanding : 1, The MultiCloud deployment instance’s NBI endpoint can be found from the cloud region in AAI (e.g. "identity-url": "http://<service<https://urldefense.proofpoint.com/v2/url?u=http-3A__-253Cservice&d=DwQGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6qPcDOqMgwf1K_r6YIIHhw&m=85aq405WfgHwU0C-y6F23V_WpTZA1WOtS4TZ3l-lzlg&s=_ZMhzvDm_TZMhLTiTfsda-hs9mMB236hnhiX3k0bGdc&e=> ip>: <service port>/api/multicloud-titanium_cloud/v0/CloudOwner2_RegionOne/identity/v2.0") 2, This identify-url are populated by multicloud plugin during the VIM/Cloud instance on boarding procedure. So it is possible that different cloud region are populated with different multicloud NBI endpoint. 3, Idealy, the VIM/Cloud instance on boarding procedure are triggered from and excuted in the context of respectively ONAP instance, so for those edge cloud orchestrated by edge ONAP will be on boarded via the edge ONAP ESR portal/API. In this case the multicloud plugin instance within the same ONAP instance will provision the cloud region with the consistent endpoints , I mean , with the same service ip :port . But the muticloud plugin instance from different ONAP instance will provision cloud region with different endpoint (I mean, different service ip: port) Does that make sense? Please let me know what you think of that. Thanks Best Regards, Bin Yang, Solution Readiness Team, Wind River Direct +86,10,84777126 Mobile +86,13811391682 Fax +86,10,64398189 Skype: yangbincs993 From: onap-discuss-boun...@lists.onap.org<mailto:onap-discuss-boun...@lists.onap.org> [mailto:onap-discuss-boun...@lists.onap.org] On Behalf Of Addepalli, Srinivasa R Sent: Wednesday, June 13, 2018 3:29 AM To: MORALES RUVALCABA, VICTOR; onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: Re: [onap-discuss] [coe] Team meeting Agenda Hi Victor and Shashank, There were few questions from Multi-Cloud meeting yesterday (Shankar and others). It would be good if we could cover following also today - K8S plugin needs to have access to information that connect to K8S masters in the remote edge-clouds/sites. Normal practice for K8S client is to read kubeconfig file that has connectivity information for each cluster and certificate/private key to be used to communicate with remote site K8S master. In ONAP, we have ESR. Are we going to use ESR or K8S plugin provides its own way to upload kubeconfig information. If so, how does get stored in A&AI. Is there any schema change required? - K8S plugin at the run time will use same deployment template files multiple times. But, each deployment instance needs to be uniquely identified. Normally, ObjectMeta.Name is used to uniquely identify each deployment. I guess it means that K8S plugin needs to generate this name dynamically to make it unique. One way to do this is to create UUID, concatenate with Name field of deployment template (artifact) and store the UUID in the A&AI on per VNF basis. Does this require any A&AI schema changes? When Delete/Update/Query VNF is called at later time, it is expected that concatenated name is passed by SO (test SO in this case). Let us discuss this too today. Thanks Srini From: onap-discuss-boun...@lists.onap.org<mailto:onap-discuss-boun...@lists.onap.org> [mailto:onap-discuss-boun...@lists.onap.org] On Behalf Of Morales, Victor Sent: Tuesday, June 12, 2018 8:47 AM To: onap-discuss@lists.onap.org<mailto:onap-discuss@lists.onap.org> Subject: [onap-discuss] [coe] Team meeting Agenda Hey there, These are the topics that I’d like to talk about today • Project status • API Definition Discussion • Opens But I want to check first if someone has an additional topic that needs to discussed. Regards, Victor Morales This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.com_about_email-2Ddisclaimer&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6qPcDOqMgwf1K_r6YIIHhw&m=kKJpI7d4BvGzsfgElthwPCjo7-CR9Z7D0x-r3mXt0KE&s=sMJvmK7Wi8SPtkpR9A39I7Q0IKKqXbq5pL8R1E-E5XI&e=> This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at https://www.amdocs.com/about/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.amdocs.com_about_email-2Ddisclaimer&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6qPcDOqMgwf1K_r6YIIHhw&m=kwpfFFX81WvykZjVkbAUPoQu4FFsAjsih1D_3eFEG-0&s=TF-I10ivjV3f7b4Z96AwgCLo3LZzD2F0I23m_0M1zog&e=>
_______________________________________________ onap-discuss mailing list onap-discuss@lists.onap.org https://lists.onap.org/mailman/listinfo/onap-discuss