Sorry I could not attend this morning. In general, I won’t be able to make it regularly due to conflicts at this time slot (Wed 8am). Because of TZ differences and other priorities for each of us, I think what would help is to track better the progress of various tasks, either through gerrit review of specs or etherpads and perhaps a bit more interaction by email? I personally find this mode of interaction more efficient and necessary to complement the GTM meetings
I have few comments on the meeting minutes related to soak test: Long-Duration testing Discussion. (SridharRao, 16:04:37) Because RFC2544 tests require traffic to be stopped and the test to Pause while the Queues empty (acm, 16:11:46) RFC2889 tests can make measurements periodically, like every 5 minutes (acm, 16:12:19) Metric in Max Frame rate, but test is conducted at Throughput level. (acm, 16:27:08) Can we use interface packet counts? (acm, 16:28:57) Martin's query - Terminating Soak Test - interface counts on DUTs? (SridharRao, 16:29:31) Difficulty is in synchronizing when the counters IN/OUT are read. (acm, 16:29:36) can test-steps be use to realize Soak tests in VSPERF? (SridharRao, 16:32:34) I wonder if alerting the operator if Loss occurs would be sufficient? (acm, 16:33:28) Answer: Possible if TGens supported APIs such as 'Stop'. (SridharRao, 16:33:35) ACTION: acm Update the SOAK test descriptions according to our current requirements (acm, 16:39:24) Questions for Christian's 2 hour test (acm, 16:42:21) ACTION: Sridhar to check with Spirent on Soak Tests using STCv. (SridharRao, 16:45:34) From my experience with NFVbench, I also found the tracking of packet drops on a long running test to be tricky. To get exact packet counts you need to stop the traffic and let queues empty – that is true for any TG (we tried with TRex and Spirent). The time for the TG to stop traffic, collect stats and resume the traffic can also be detrimental – this delay varies based on the TG, I know the Spirent API from python is slow to react to that sequence (several seconds) while TRex API is quite a bit faster but still around a second (as you need to let queues empty). Regardless, it looks like stop and go with multi-second pauses between runs during long duration test might not be acceptable. Another approach I have experimented is to let the traffic run non-stop and collect RX and TX packet stats at intervals (say every few seconds). Because the time to get the TX counter and RX counter is never identical, diffing TX and RX will not be very accurate and may sometimes even yield negative values (RX counter > TX counter). However the trend of the 2 series will still be reasonably usable: if the actual drop rate is non zero, over time the packet counters will diverge sufficiently to overcome the counter read time discrepancy, while if the actual drop rate is zero, you would expect the gap to be generally constant. Using that approach a long duration run can produce pairs of TX/RX counters at interval, which can then be plotted in real time using tools like Grafana, any packet drop will show over time with 2 diverging lines while no drops would show 2 roughly parallel lines. How early it will show depends on the actual drop rate and the RX/TX counter “error” margin (which itself is a function of the current rate and actual time difference between the read of the 2 counters in the TG). At the end of the run you can always get the exact RX/TX counts + the exact average drop rate. I don’t quire remember how fast was the RX/TX read API during traffic with Spirent but the TRex API was pretty fast at that (easily sub-second). I also found interesting to also track the latency histogram at intervals (using rolling windows) but I’m not sure TGs can support that. For the long duration test, it is currently possible to run NFVbench for long stretches and get the detailed results with RX/TX interval reports as mentioned above. Other points/questions: * Any update on the traffic gen API split into a new repo? * Did the VSPERF team submit any presentation at the coming ONS summit in LA (March)? I submitted one with Al centered around Full system NFVi data plane benchmarking and benchmarking standards evolution – more targeted to SP/devops. We’ll know by February if we made it. * I would have liked the VSPERF and test-wg community to provide some feedback on the role of data plane “validation” in OVP, so far we only had comments from Eddie and me * I need to find time to present to the test-wg/vsperf meetings some thoughts about how to move forward in latency histogram implementations, something to put under the radar and that I’d be happy to discuss over email. I think we should be able to implement the reporting of rolling latency histograms during runs. Regards, Alec From: "Rao, Sridhar" <sridhar....@spirent.com> Date: Wednesday, January 17, 2018 at 8:52 AM To: "opnfv-tech-discuss@lists.opnfv.org" <opnfv-tech-discuss@lists.opnfv.org> Cc: "'ALFRED C 'MORTON (AL)''" <acmor...@att.com>, 'Trevor Cooper' <trevor.coo...@intel.com>, "'Mars Toktonaliev (Nokia - US/Irving)'" <mars.toktonal...@nokia.com>, 'Cian Ferriter' <cian.ferri...@intel.com>, "Martin Klozik (martinx.klo...@intel.com)" <martinx.klo...@intel.com>, "Christian Trautman (ctrau...@redhat.com)" <ctrau...@redhat.com>, "Bill Michalowski (bmich...@redhat.com)" <bmich...@redhat.com>, "'Elias, RichardX'" <richardx.el...@intel.com>, "Alec Hothan (ahothan)" <ahot...@cisco.com> Subject: RE: [vsperf] Agenda for VSPERF weekly meeting - 17 Jan 2018 (ww52) Meeting Minutes can be found here: http://ircbot.wl.linuxfoundation.org/meetings/opnfv-vswitchperf/2018/opnfv-vswitchperf.2018-01-17-16.02.html Thanks for the interesting discussion on long-duration tests. Next week we will be discussing Traffic-Profile in VSPERF – Current and Future. Alec, if you can also make it, if you be great! Regards, Sridhar K. N. Rao (Ph. D) Architect +91-9900088064 From: Rao, Sridhar Sent: Wednesday, January 17, 2018 10:01 AM To: 'opnfv-tech-discuss@lists.opnfv.org' <opnfv-tech-discuss@lists.opnfv.org> Cc: 'ALFRED C 'MORTON (AL)'' <acmor...@att.com>; 'Trevor Cooper' <trevor.coo...@intel.com>; 'Mars Toktonaliev (Nokia - US/Irving)' <mars.toktonal...@nokia.com>; 'Cian Ferriter' <cian.ferri...@intel.com>; Martin Klozik (martinx.klo...@intel.com) <martinx.klo...@intel.com>; Christian Trautman (ctrau...@redhat.com) <ctrau...@redhat.com>; Bill Michalowski (bmich...@redhat.com) <bmich...@redhat.com>; 'Elias, RichardX' <richardx.el...@intel.com>; 'Alec Hothan (ahothan)' <ahot...@cisco.com> Subject: [vsperf] Agenda for VSPERF weekly meeting - 17 Jan 2018 (ww52) Hi All, Agenda for this week’s meeting: 1. Development Update. 2. Discussion – Topic: VSPERF-Long Duration Testing Meeting minutes · WW51 http://ircbot.wl.linuxfoundation.org/meetings/opnfv-vswitchperf/2018/opnfv-vswitchperf.2018-01-10-16.00.html · WW50 http://ircbot.wl.linuxfoundation.org/meetings/opnfv-vswitchperf/2017/opnfv-vswitchperf.2017-12-20-16.01.html · WW49 http://ircbot.wl.linuxfoundation.org/meetings/opnfv-vswitchperf/2017/opnfv-vswitchperf.2017-12-13-16.00.html -------------------------------------------------------- Time: Wednesday UTC 16h00, Ireland (UTC) 16h00, PST 8h00 (UTC/GMT -8) IRC: freenode https://freenode.net/ channel: #opnfv-vswitchperf http://webchat.freenode.net/?channels=opnfv-vswitchperf Audio: https://global.gotomeeting.com/join/391235029 You can also dial in using your phone. United States: +1 (571) 317-3116 Access Code: 391-235-029 Phone numbers from other countries are here … https://wiki.opnfv.org/display/meetings Regards, Sridhar K. N. Rao (Ph. D) Architect +91-9900088064 Spirent Communications e-mail confidentiality. ------------------------------------------------------------------------ This e-mail contains confidential and / or privileged information belonging to Spirent Communications plc, its affiliates and / or subsidiaries. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution and / or the taking of any action based upon reliance on the contents of this transmission is strictly forbidden. If you have received this message in error please notify the sender by return e-mail and delete it from your system. Spirent Communications plc Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Tel No. +44 (0) 1293 767676 Fax No. +44 (0) 1293 767677 Registered in England Number 470893 Registered at Northwood Park, Gatwick Road, Crawley, West Sussex, RH10 9XN, United Kingdom. Or if within the US, Spirent Communications, 27349 Agoura Road, Calabasas, CA, 91301, USA. Tel No. 1-818-676- 2300
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
