Hi Alec, We had a section in the OPNFV Plugfest portion already... It was under Hackfest, but given that there was testing, I moved it into the Plugfest section. See if it looks OK. Feel free to put comments directly in the document <https://docs.google.com/document/d/17xSPe7BTOq2LMa-qHLiBfnOSo6JXhvqxjriMZUKhcrI/edit?ts=5b8843df#> .
Thanks, Amar *Comparison of Performance Testing ToolsFive OPNFV projects use the traffic generators. However, differences in traffic generator drivers can cause performance result variations between these projects even when measuring the same system under test (SUT). An experiment to test the same L2 forwarding OVS-DPDK vSwitch across VSPERF, NFVBench, and an additional benchmarking tool was conducted. The conclusion of this experiment is that the packet format (such as the number of flows created) and packet path can have a significant impact on results. When the exact same conditions were applied, all three projects produced the same results. This and ongoing future experiments could lead to future efforts around developing a common driver or set of libraries, harmonizing requirements around traffic generators — while taking in real-life considerations.* On Thu, Aug 30, 2018 at 7:26 AM, Alec Hothan (ahothan) <ahot...@cisco.com> wrote: > > > There was a joint workgroup to compare 3 different benchmarking tool on > the system under test used by VSPERF that is running an OVS-DPDK vswitch. > > The 3 tools were: VSPERF, NFVbench, External tool by Redhat > > The goal was to: > > - See how different tools differ in their results when measuring the > exact same system under test > - Verify that NFVbench and other external tool can run on a new system > with just a vswitch and no openstack > > The conclusion is that > > - all 3 tools could run and measure the performance of the same system > and produce similar results when test conditions are aligned properly. > - Significant differences can be measured if test conditions differ > even slightly (e.g. number of flows) > - NFVbench and the external tool can measure a non-openstack bafe bone > vswitch > > > > > > The new section looks good otherwise. > > > > Thanks, > > > > Alec > > > > > > *From: *<opnfv-tech-discuss@lists.opnfv.org> on behalf of David McBride < > dmcbr...@linuxfoundation.org> > *Date: *Wednesday, August 29, 2018 at 12:04 PM > *To: *"tim.irn...@ericsson.com" <tim.irn...@ericsson.com>, Mingjiang Li < > rexlee8...@gmail.com>, "MORTON, ALFRED C (AL)" <acmor...@att.com>, Georg > Kunz <georg.k...@ericsson.com>, "Foley, Emma L" <emma.l.fo...@intel.com> > *Cc: *TECH-DISCUSS OPNFV <opnfv-tech-discuss@lists.opnfv.org>, Amar > Kapadia <akapa...@aarnanetworks.com>, "pierre.ly...@keysight.com" < > pierre.ly...@keysight.com>, Heather Kirksey <hkirk...@linuxfoundation.org>, > Lincoln Lavoie <lylav...@iol.unh.edu> > *Subject: *Re: [opnfv-tech-discuss] [plugfest][fraser] review of OPNFV > plugfest report (part of joint ETSI/OPNFV report) > > > > Team, > > > > We need your input on this document once again. A new section titled > "Joint Testing Activities" has now been integrated. If you would, please > take a few moments to review and provide feedback on just that section. To > repeat, we only need feedback on the section titled "Joint Testing > Activities". Please complete your review by Sept 7, 5 p.m. (PT). > > > > Let me know if you have any questions. Thank you. > > > > David > > > > On Fri, Jul 6, 2018 at 5:19 PM David McBride <dmcbr...@linuxfoundation.org> > wrote: > > Team, > > > > Thanks for agreeing to review the OPNFV plugfest portion > <https://docs.google.com/document/d/17xSPe7BTOq2LMa-qHLiBfnOSo6JXhvqxjriMZUKhcrI/edit?usp=sharing> > of the joint ETSI/OPNFV report. Your feedback is invaluable in ensuring > that the document is accurate, comprehensive, and useful for our > community. We will be accepting comments from now until July 20. > > > > Note that this material that you will be reviewing is part of a larger, > joint report between OPNFV and ETSI. This joint report will also include a > section, currently in development, on OPNFV and ETSI collaboration during > the plugtest/plugfest in June. I will alert you when that section becomes > available, so that you may review it, as well. > > > > Finally, I'd like to thank Amar for his hard work documenting plugfest > activities, and developing and editing the document. > > > > Please let me know if you have any questions. > > > > David > > > > -- > > *David McBride* > > Release Manager, OPNFV > > Mobile: +1.805.276.8018 > > Email/Google Talk: dmcbr...@linuxfoundation.org > > Skype: davidjmcbride1 > > IRC: dmcbride > > > > > -- > > *David McBride* > > Release Manager, OPNFV > > Mobile: +1.805.276.8018 > > Email/Google Talk: dmcbr...@linuxfoundation.org > > Skype: davidjmcbride1 > > IRC: dmcbride > > > -- Amar Kapadia Aarna Networks, Inc. (408)390-0951 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21897): https://lists.opnfv.org/g/opnfv-tech-discuss/message/21897 Mute This Topic: https://lists.opnfv.org/mt/23179182/21656 Group Owner: opnfv-tech-discuss+ow...@lists.opnfv.org Unsubscribe: https://lists.opnfv.org/g/opnfv-tech-discuss/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
