Hi xudan,
I agree with you, if someone really wants to, he well always find a way to cheat especially if the entire code is open source. Considering this, the only reason for the entire certification topic is for operators to have a way of pinning down the VNF provider on the ONAP compliance when making a contract. Still the certificate would be worth nothing really. A certificate in my understanding is something I can trust. For the OPNVF/ONAP certificate this is not given at all. Even if I get a ONAP certified VNF I still must make my own retest to assure, that it really is ONAP compliant. Where is the difference of having no certificate but providing the validation software so everyone can check themselves if a VNF is ONAP compliant? In my opinion the only way to get a trustworthy certification process is by moving the tests of the VNF to the LFN and automatically check the results. Then if you're worried about storing the VNF package due to security delete the VNF and only keep the results, a hash and the certificate in the database. I do not feel like this adds a lot of complexity to the process from the VNF providers perspective. Currently he must log onto a web portal and upload results. For the remote option he must log onto a web portal and upload the VNF package. But for the operator it removes a lot of trouble, because the certificate is trustworthy, and you do not have to assume that the provider cheated and retest everything on your own. I am pretty new to the topic, so maybe I am missing something. What are the cons of a LFN hosted validation platform? Best regards Vincent Von: xudan (N) <[email protected]> Gesendet: Montag, 11. Februar 2019 09:55 An: LOVETT, TREVOR J <[email protected]>; Lincoln Lavoie <[email protected]> Cc: [email protected]; Gaoweitao (Victor, Cloudify Network OSDT) <[email protected]>; [email protected]; [email protected]; Kanagaraj Manickam <[email protected]>; STARK, STEVEN <[email protected]>; WRIGHT, STEVEN A <[email protected]>; Katsaounis Molyvas Stamatios <[email protected]>; HALLAHAN, RYAN <[email protected]>; Scharf, Vincent <[email protected]> Betreff: RE: [opnfv-tech-discuss] CVC Jonit Meeting (Feb 4th) Agenda Hi Trevor and Lincoln, I'd like to share some experience from OPNFV with you. OVP had the same problem when we began to design its framework. How to ensure the uploaded results are trusted? How to make sure the Dovetail tool hasn't been changed locally by testers? Then we designed a workflow with signature and checksum of all the source code and result files. The attachment is the signature process. But at last, OVP decide to give up all these methods which try to make it as credible as possible. Because 1. We don't think there can be a perfect system to avoid this. If someone want, he/she can always find ways to get the fraudulent report and upload it. 2. All result files are related to each other. No one can make it totally undetectable after modifying some files as the results are open for all reviewers. 3. The testing tools are totally open source and public for everyone. It's easy for everyone including Providers to retest any SUT/VNF. It's meaningless for vendors to tamper with the results to just get the verified badge. Considering these, OVP abandoned all the signature and encrypt methods and make the workflow as simple as possible. Cheers, Dan From: [email protected] <mailto:[email protected]> [mailto:[email protected]] On Behalf Of LOVETT, TREVOR J Sent: Friday, February 08, 2019 4:04 AM To: Lincoln Lavoie <[email protected] <mailto:[email protected]> > Cc: [email protected] <mailto:[email protected]> ; Gaoweitao (Victor, Cloudify Network OSDT) <[email protected] <mailto:[email protected]> >; [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> ; Kanagaraj Manickam <[email protected] <mailto:[email protected]> >; STARK, STEVEN <[email protected] <mailto:[email protected]> >; WRIGHT, STEVEN A <[email protected] <mailto:[email protected]> >; Katsaounis Molyvas Stamatios <[email protected] <mailto:[email protected]> >; HALLAHAN, RYAN <[email protected] <mailto:[email protected]> >; [email protected] <mailto:[email protected]> Subject: Re: [opnfv-tech-discuss] CVC Jonit Meeting (Feb 4th) Agenda Thanks for describing the OPNFV flow. That's helpful. With regards to Option 1, I would suggest separating "where the tests are run" from "how the results are certified". I don't think having the provider upload the artifacts (Heat or CSAR) with their results necessarily constrains or bifurcates their testing experience. They can still run these tests in the lab and produce their results. In the infrastructure or functional testing world recreating the testing is prohibitive. I'm sure that was a large contributor to the certification scheme (independent review of the logs) used in OPNFV. In this case, we're performing rather simple, repeatable scans of static artifacts. The provider can still develop and test externally producing their result files. However, for this portion of the results we don't really have to rely on simple human inspection of forgeable text results - we can just retest them if they provide the associated artifact. The only change would be uploading the artifacts along with the results. Now there might be other reasons we deem uploading the artifacts prohibitive, but in that example the actual testing experience has not changed. We're just asking for one additional artifact during result submission. To elaborate on the checksum scenario in Option 2, the checksum is a unique, repeatable value generated from the contents of the artifact (all files). Any change to the artifact would generate a different checksum. If a provider submitted results and a checksum, they are stating that a specific artifact (identifiable by the checksum) passed these tests in the result file. The OVP portal or human reviewers can't verify the checksum - they're just looking at the result text files which can always be fabricated no matter what we do. However, the eventual consumer of the VNF can validate it. When they receive the artifact, they can run the validations independently. The results and checksums should match. If either is different, then either the results were doctored or they have provided a different artifact to the consumer (different checksum). Happy to discuss further on the next call. I'm sure we can work out a solution. Thanks, Trevor From: Lincoln Lavoie [mailto:[email protected]] Sent: Thursday, February 07, 2019 11:06 AM To: LOVETT, TREVOR J <[email protected] <mailto:[email protected]> > Cc: [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> ; Kanagaraj Manickam <[email protected] <mailto:[email protected]> >; STARK, STEVEN <[email protected] <mailto:[email protected]> >; WRIGHT, STEVEN A <[email protected] <mailto:[email protected]> >; Katsaounis Molyvas Stamatios <[email protected] <mailto:[email protected]> >; HALLAHAN, RYAN <[email protected] <mailto:[email protected]> >; [email protected] <mailto:[email protected]> Subject: Re: CVC Jonit Meeting (Feb 4th) Agenda Hi Trevor, The work flow is, user runs the dovetail tool, which produces the log output the user can upload to the portal. This allows users to run internal testing, debugging, etc. When they are ready to apply for certification, they upload the results to the portal, along with submitting an application (i.e. info about the VNF, like marketing name, etc.). That application and logs are reviewed by the reviewers (can't be from the same company). It takes two approvals from two different reviewers to pass that gate (i.e. 2 sets of eyes on the results). Once approved, the admin is able to publish the listing to the website, etc. Hopefully this clarifies things on the work flow. For option 2, the challenge is, couldn't they just paste over the checksum before they submit the results? Cheers, Lincoln On Thu, Feb 7, 2019 at 11:57 AM LOVETT, TREVOR J <[email protected] <mailto:[email protected]> > wrote: What was the solution in OPNFV? Can someone lay out the proposed experience and flow as well? Based on prior conversations, I thought the proposal for a short-term solution was a human-being from the VNF Provider/Lab to upload the result files to the web site. If there's an alternate proposed flow (i.e. Dovetail directly uploading the results to the portal), then I think it would be helpful to start getting that written down somewhere and it has bearing on how the results are secured/certified. It's not clear that all players are on the same page here at this juncture. In Option 2, I don't fully understand your comment about a clever user overwriting what is in the logs. I did state that the report itself could still be fabricated so I'm not arguing that it couldn't be. My point was that if they provide a checksum with their report, then the result could at least be challenged and verified. If a consumer of the VNF can't reproduce the test results on the files with the same checksum then the fraud can be detected. Thanks, Trevor From: Lincoln Lavoie [mailto: <mailto:[email protected]> [email protected]] Sent: Thursday, February 07, 2019 10:22 AM To: LOVETT, TREVOR J < <mailto:[email protected]> [email protected]> Cc: <mailto:[email protected]> [email protected]; <mailto:[email protected]> [email protected]; <mailto:[email protected]> [email protected]; <mailto:[email protected]> [email protected]; Kanagaraj Manickam < <mailto:[email protected]> [email protected]>; STARK, STEVEN < <mailto:[email protected]> [email protected]>; WRIGHT, STEVEN A < <mailto:[email protected]> [email protected]>; Katsaounis Molyvas Stamatios < <mailto:[email protected]> [email protected]>; HALLAHAN, RYAN < <mailto:[email protected]> [email protected]>; <mailto:[email protected]> [email protected] Subject: Re: CVC Jonit Meeting (Feb 4th) Agenda Hi Trevor, I think one concern I have with Option 1 is, it will only work for the VVP compliance testing, as soon as we move towards more advanced testing (i.e. live cycle testing, etc), it would cause a split path for VNF testing, where you have to do 1/2 of your testing through a web portal and then 1/2 on some type NFVI / MANO platform that is in your lab or another lab. I think this will just make for a complicated future. For option 2, this provides some protection, but a clever user could always just overwrite what is in the logs before submitting this. Many of these questions / points were already talked about a lot within the OPNFV program, when that was developed and there was a desire to protect the results as well. Cheers, Lincoln On Thu, Feb 7, 2019 at 10:42 AM LOVETT, TREVOR J <[email protected] <mailto:[email protected]> > wrote: During the call, one item that came up was how to lessen the likelihood of a fraudulent report being submitted as part of the certification/badging process. One suggestion was to include additional runtime, vnf-specific trace information in the report itself making it more difficult to fabricate. The VVP team discussed this, but we feel this would be non-trivial to implement and not truly prevent the core issue. We would not be in favor of pursuing this option. If we feel that we need to protect against fraudulent reports, then we are suggesting two alternatives that can be discussed in a future session. 1) Certification Portal Executes the validations directly using Dovetail: a. Since we are only scanning a set of static files, and the validation tools will be available in a container for execution, we could simply have the portal run the scan upon uploading of the artifacts. This eliminates a fraudulent report submission entirely, and avoids other issues such as ensuring the proper versions of the tools are executed externally. b. I do understand there may be sensitivity to uploading the proprietary artifacts, but there is no need for them to be stored or shared indefinitely. They would only need to be retained for the duration of the scan. After that we simply need to retain the checksum of the artifact that was submitted. c. If we simply don't think VNF Providers would agree to that, then our back-up option is below 2) Display Artifact Checksum Along with Badge a. The VVP tool already computes an MD5 hash of all the file contents and stores that in the report b. If we retain this information and publish it on the certification site, then the ultimate consumer of the artifact can at least verify the artifact they receive from the vendor corresponds to the artifact submitted for certification. We should likely do this even in option #1 for the same reason. c. This doesn't mean prevent the provider from submitting a fabricated report, but it does ensure that it can be challenged and verified after-the-fact if needed. d. I believe the CSAR similarly has an checksum that can be used for this purpose as well. We can discuss the topic further in one of our upcoming joint calls. Thanks, Trevor From: [email protected] <mailto:[email protected]> [mailto:[email protected] <mailto:[email protected]> ] On Behalf Of Gaoweitao(Victor) Sent: Sunday, February 03, 2019 2:26 AM To: [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> ; [email protected] <mailto:[email protected]> Cc: Kanagaraj Manickam <[email protected] <mailto:[email protected]> >; Lincoln Lavoie <[email protected] <mailto:[email protected]> > Subject: [onap-discuss] CVC Jonit Meeting (Feb 4th) Agenda Hi VNFSDK/VVP/Dovetail Developers, Here is the initial agenda for Feb 4 <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_display_ DW_CVC-2BJoint-2BMeeting-2B02-2D04-2D2019&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg& r=dIU_U39dl9FoORwHk72kyMcMyxm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5 R5GeOb-B0URqdcI&s=-YhkEGJSSKD46SqQByjR_bJvnYWC7lsI62elfSWGTQA&e=> th Joint meeting, feel free to add your topics: 1. CVC one click deployment 2. Show marketplace capability and is it possible used for dovetail test script execution trigger? I will be absence due to Chinese new year and Kanagraj from VNFSDK Team is my proxy and host the meeting. The zoom bridge is still available during my absence: https://zoom.us/j/346625009 <https://urldefense.proofpoint.com/v2/url?u=https-3A__zoom.us_j_346625009&d= DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORwHk72kyMcMyxm1s8RqOhr8grdzE2 s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s=-wmrOsLWyh45frQOsVJCAqGlh2 DmMNJu15PMacF17Z0&e=> The previous meeting minutes is here: https://wiki.onap.org/display/DW/Joint+CVC+Meeting <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.onap.org_display_ DW_Joint-2BCVC-2BMeeting&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORw Hk72kyMcMyxm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s =mkSu7O2_kzmJipm2xlbfaaZGwXViIQJvQt12MmFifJo&e=> BR Victor _._,_._,_ _____ Links: You receive all messages sent to this group. View/Reply Online (#15324) <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_g_onap- 2Ddiscuss_message_15324&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORwH k72kyMcMyxm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s= 2DIdFBAfxs43n6h4Lw6yH2FGyRmBqimlbz41vQNgGtw&e=> | Reply To Group <mailto:[email protected]?subject=Re:%20%5Bonap-discuss%5D%20CVC%2 0Jonit%20Meeting%20%28Feb%204th%29%20Agenda> | Reply To Sender <mailto:[email protected]?subject=Private:%20Re:%20%5Bonap-discuss%5D%20 CVC%20Jonit%20Meeting%20%28Feb%204th%29%20Agenda> | Mute This Topic <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_mt_2963 8769_1198157&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORwHk72kyMcMyxm 1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s=LeTLkAhNSX8 sD_rxDuVs1mIlWHYZESMZPmuIf18ic_A&e=> | New Topic <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_g_onap- 2Ddiscuss_post&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORwHk72kyMcMy xm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s=rxFGHYc7s Nu0tzbdyYovrN7Kpnj-T_bZ9Gu0niWSMT8&e=> Your Subscription <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_g_onap- 2Ddiscuss_editsub_1198157&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoOR wHk72kyMcMyxm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI& s=6n1a-52slPnkgXiGGJySGaZzkxq_t5TrSTw_o9GtvqI&e=> | Contact Group Owner <mailto:[email protected]> | Unsubscribe <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.onap.org_g_onap- 2Ddiscuss_unsub&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=dIU_U39dl9FoORwHk72kyMcM yxm1s8RqOhr8grdzE2s&m=ZKQoo454X4U_lELjzO_WCevaKAV5R5GeOb-B0URqdcI&s=x3ICz0g1 PsytMM03urA8OHzkY-WW_yA77jqC4tU5b7Q&e=> [[email protected] <mailto:[email protected]> ] _._,_._,_ -- Lincoln Lavoie Senior Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 [email protected] <mailto:[email protected]> https://www.iol.unh.edu <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iol.unh.edu&d=DwMF aQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=g9LhwjMTPM4AuoWvYyDmqA&m=UdvYRI0iWQVoM8OjcQ2mv Jst7GbGwawBimVeis3IEm4&s=v7yOqkd93k2wkmp4FMkKZDTKNEblZrYoEpkpmHCrM5c&e=> +1-603-674-2755 (m) <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iol.unh.edu_&d=DwM FaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=g9LhwjMTPM4AuoWvYyDmqA&m=UdvYRI0iWQVoM8OjcQ2m vJst7GbGwawBimVeis3IEm4&s=M3cJfFEQvxci-Zj1_7zb_oIPoAuzKoUV4mXBTLeOcRc&e=> -- Lincoln Lavoie Senior Engineer, Broadband Technologies 21 Madbury Rd., Ste. 100, Durham, NH 03824 [email protected] <mailto:[email protected]> https://www.iol.unh.edu <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iol.unh.edu&d=DwMF aQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=g9LhwjMTPM4AuoWvYyDmqA&m=ASzlsco1NQ_ij_9fuiLk- _y3RaYqHeyy9UrPQsEGMnk&s=yw5oCAAUBEh4sb3ycD5QfDXWKBYTquqHlCVFVwJyPWE&e=> +1-603-674-2755 (m) <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.iol.unh.edu_&d=DwM FaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=g9LhwjMTPM4AuoWvYyDmqA&m=ASzlsco1NQ_ij_9fuiLk -_y3RaYqHeyy9UrPQsEGMnk&s=qs5TA4zmA7QSGWAfTZsKnUCecW27hp_JoxNCXNxkBNQ&e=>
smime.p7s
Description: S/MIME cryptographic signature
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#22799): https://lists.opnfv.org/g/opnfv-tech-discuss/message/22799 Mute This Topic: https://lists.opnfv.org/mt/29667350/21656 Group Owner: [email protected] Unsubscribe: https://lists.opnfv.org/g/opnfv-tech-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
