Hi, My understanding is that the discussion was heading for the following:
* A means to collect data from ONAP and export it to a data lake. This would be part of the “ONAP definition” when all is complete. The idea here was the use of DMaaP as a starting point, and I understand this was heading towards going in the DCAE project. * Then it is required to test it. For there some form of reference data lake would be included in integration to test the data. The point being that this approach allows for different data lakes without ONAP specifying what that is, and can still allow experimental work in the integration. I think as this hasn’t been approved, it isn’t on Catherines list. BR, Steve From: onap-tsc@lists.onap.org <onap-tsc@lists.onap.org> On Behalf Of Vijay VK Sent: Thursday 31 January 2019 18:26 To: onap-tsc@lists.onap.org Subject: Re: [onap-tsc] Data Lake Hi TSC, Could someone please clarify what is the scope of datalake POC targeted for Dublin? Also I don’t see this tracked under current requirement - https://wiki.onap.org/display/DW/Dublin+Release+Requirements? Regards, Vijay From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> On Behalf Of Kenny Paul Sent: Thursday, January 31, 2019 8:56 AM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: Re: [onap-tsc] Data Lake Actually comparing the vote at the meeting to the attendance at the meeting, datalake as a POC only for Dublin passed. Thanks! -kenny From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> On Behalf Of Lingli Deng Sent: Wednesday, January 30, 2019 10:16 PM To: onap-tsc <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> Subject: Re: [onap-tsc] Data Lake Hi TSC, As for Datalake proposal, I suppose it is well discussed and understood. I recalled the consensus was it is useful and we want it to be done, but not including the backend storages and as part of DCAE for Dublin. Kenny also initiated a vote in the chatting room during the TSC call two weeks ago. Is that conclusive or do we need another vote to settle the issue and move on? Thanks, Lingli/邓灵莉 中国移动通信研究院 denglin...@chinamobile.com<mailto:denglin...@chinamobile.com> ----邮件原文---- 发件人："ramki krishnan via Lists.Onap.Org" <ramkik=vmware....@lists.onap.org<mailto:ramkik=vmware....@lists.onap.org>> 收件人："onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>" <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> 抄 送: (无) 发送时间：2019-01-20 07:27:04 主题：Re: [onap-tsc] Data Lake Well described Srini. A clarification – the Kafka broker is installed as part of ONAP correct? Since DMaaP is a wrapper over Kafka, I am more inclined towards the datalake connectivity enablement being part of DMaaP project. Thanks, Ramki From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> On Behalf Of Srini Sent: Saturday, January 19, 2019 1:11 PM To: 'onap-tsc@lists.onap.org' <onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org>> Subject: Re: [onap-tsc] Data Lake Just to add some more information: · Apache Kafka added Authorization and ACLs functionality (http://kafka.apache.org/documentation/#securoty_authz<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fkafka.apache.org-252Fdocumentation-252F-2523securoty-5Fauthz-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DCcCqJEHrx8G1OfRG-252Bz6Fskos0POvRF3RZ4KaTU-252FTD2c-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=Bmc2Bt5jZI8BKnJCiQyTN4cX9Dzcf5efYgBp8m2cHE8&e=>) which allows providing configurable access control to external entities that need access to Kafka topics.. · Kafka Operator (https://github.com/strimzi/strimzi-kafka-operator<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fgithub.com-252Fstrimzi-252Fstrimzi-2Dkafka-2Doperator-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DasGgFuZAnpqltDsjTuATgnGvrRkAbdJMXj59FSs50vQ-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=yEMPBvdGp_sBXrOpUaBo2JoyltXvUtCun4O8xy-Odxo&e=>) made it simple to create ACLs using Kubernetes resources. Both of above technologies/features can be added. From the workflow perspective, there are two operations by the ONAP user: · Register external Kafka entity (In this case, external data like is a kafka client) · Unregister the entity. It is similar to the way cloud-regions are registered via ESR. Records can be added to A&AI DB. One can provide GUI/CLI to make it simple for ONAP-users to add these in A&AI. As part of registration, one can add users or certificate names and topic they are allowed to subscribe to. When external entity registered in A&AI, it can trigger the kafka operator to add the ACL records. With this, each data lake can be brought up outside of ONAP and register that with A&AI by ONAP-User. Once that happens, data lake can get hold of any messages produced on the topic and can store in its databases and provide any analytics. Thanks Srini From: Addepalli, Srinivasa R Sent: Thursday, January 17, 2019 2:12 PM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: RE: Data Lake My view is that data lakes (this one or any other data lake) can be considered as external entities (can be brought up ONAP like any other VNFs or via some other mechanisms). That includes feeder and UI. From ONAP perspective, we need to ensure that Kafka bus is accessible by third party systems, that is, authenticated registration of third party systems and RBAC on topics etc… Since this datalake is specific to ONAP monitoring, I guess it is required that all modules send all its telemetry data and transaction data to Kafka. In my view, ideal thing is to do this integration from ISTIO side car. Thanks Srini From: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> [mailto:onap-tsc@lists.onap.org] On Behalf Of Stephen Terrill Sent: Thursday, January 17, 2019 1:19 PM To: onap-tsc@lists.onap.org<mailto:onap-tsc@lists.onap.org> Subject: [onap-tsc] Data Lake Hi All, During the TSC discussion today on the data lake; the discussion was leading towards considering the data lake feeder as part of DCAE. Upon further reflection, I was wondering whether we should also consider including it as part of DMAAP as an alternative option. Just for consideration. BR, Steve [http://mediabank.ericsson.net/internet-media/Email_logo_Ericsson.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fwww.ericsson.com-252F-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DwepsxRxsQ31CVOf1Nozkd4PG1yiVn1A-252B9ycnFYty1WA-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=i5YX8OXZkhFRyaSC7bsrN2VOp29edOBRjIlI_XspoSU&e=> Stephen Terrill Senior Expert, Automation and Management TECHNOLOGY SPECIALIST BDGS RDP Architecture & Technology Phone: +34913393005 Mobile: +34609168515 stephen.terr...@ericsson.com<mailto:stephen.terr...@ericsson.com> Ericsson C/ Via de los Poblados 13. B 28033,Madrid, Madrid Spain ericsson.com<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fwww.ericsson.com-252F-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DwepsxRxsQ31CVOf1Nozkd4PG1yiVn1A-252B9ycnFYty1WA-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=i5YX8OXZkhFRyaSC7bsrN2VOp29edOBRjIlI_XspoSU&e=> [http://mediabank.ericsson.net/internet-media/Email_Message.gif]<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fwww.ericsson.com-252Fcurrent-5Fcampaign-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DZUNHI1JMl39HqSTwHVswyS6mdTexNhs3T6UKG-252B-252BUkaU-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=KDE2_U0RA211xGT3DPILe_QKTmUtmYGuMTmhdM3bPFw&e=> Our commitment to Technology for Good<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fwww.ericsson.com-252Fthecompany-252Fsustainability-2Dcorporateresponsibility-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3D-252Balc2gTkEfpmxMogi8S0flEmlFyfPCuuS4LL4B838UI-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=Y6LsT11pUj4pnelWjqCdi3r6yvhLVMmJRhpxh68GlFY&e=> and Diversity and Inclusion<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttp-253A-252F-252Fwww.ericsson.com-252Fthecompany-252Fdiversity-2Dinclusion-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3Dm1PMx8T-252FE-252BqB5JFx9QWUGyCc4zbGhTpOr81rHvea1m0-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=4X0Ky2H5z5NkVrP0_2dPuFYwsETjLtnPffGq2dG0Xts&e=> contributes to positive change. Follow us on: Facebook<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.facebook.com-252Fericsson-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3D59Ygda3ATQ3CJOI9BOUJCeNDoERf-252BVFjiV05scLlHwc-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=eqgpqZAtpYNWWP0bd5ZZNFRElvpsYkz3Jljchb999yc&e=> LinkedIn<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.linkedin.com-252Fcompany-252Fericsson-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DviiU4ApSDrLjOfYPIJB4ghm-252FRK9fEoj3sjHdGSgGeJg-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=5-l94uhUBmAYXZlFjBNPJ68bxeUyTjlFTIPUWG7Nef4&e=> Twitter<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Ftwitter.com-252FEricsson-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3Dpdtc-252FGQHTW-252BsDV5AN-252BtOF6gDqxhTfNDfZiF9K3bXb-252Bw-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=Tfa832WtU0tPAsaref0UAft4XOrgiMYiD2VihWolC3M&e=> Legal entity:ERICSSON AB registration number 556056-6258, registered office in Stockholm. This communication is confidential. Our email terms: www.ericsson.com/en/legal/privacy/email-disclaimer<https://urldefense.proofpoint.com/v2/url?u=https-3A__na01.safelinks.protection.outlook.com_-3Furl-3Dhttps-253A-252F-252Fwww.ericsson.com-252Fen-252Flegal-252Fprivacy-252Femail-2Ddisclaimer-26data-3D02-257C01-257Cramkik-2540vmware.com-257C0be423483c4f46a4e33208d67e52b18b-257Cb39138ca3cee4b4aa4d6cd83d9dd62f0-257C0-257C0-257C636835291004260562-26sdata-3DELfd-252Bee-252B2LCb7Cu-252F1qdGgAUQcom8XlihwzBTROycY3k-253D-26reserved-3D0&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=6WYcUG7NY-ZxfqWx5MmzVQ&m=BdFSnRoAwjtplUQ-5mUoSO2MnJjsgXCA7aKte2BhAXg&s=7BZ9D_gey8egfJ7tHyKJeNyS_mRdXSrlHmDKBT0_RzY&e=> -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#4552): https://lists.onap.org/g/onap-tsc/message/4552 Mute This Topic: https://lists.onap.org/mt/29189968/21656 Group Owner: onap-tsc+ow...@lists.onap.org Unsubscribe: https://lists.onap.org/g/onap-tsc/leave/2743226/1412191262/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
