[vpp-dev] Bumping dpdk to 22.11 in vpp 23.02 ?

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi,

On CSIT call just now Kai made us aware of issues with above (cryptodev, sat), 
as captured in this patch:

37840: dpdk: make impact to VPP for changes in API for DPDK 22.11 | 
https://gerrit.fd.io/r/c/vpp/+/37840

23.02 RC1 is next week and in CSIT we start testing at RC1 milestone, so it’s 
very last minute …

Also, in the past we got burned by DPDK bump requiring bumping firmware 
versions on FVL and CVL NICs, which for our performance testbed fleet is a bit 
of an operation (e.g. on Arm we have to remove the NICs and put them into Xeon 
machines to do firmware upgrade, unless things improved recently).

Asking for views if we could delay dpdk ver bumping to avoid rushing it in, 
especially that there are open issues?

Thoughts?

Cheers,
-Maciek
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22448): https://lists.fd.io/g/vpp-dev/message/22448
Mute This Topic: https://lists.fd.io/mt/96202380/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [tsc-private] [csit-dev] TRex - replacing CVL with MLX for 100GbE

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi Dave,

Per my action from the last week TSC meeting (item 4.c. in [1]), here is
the list of HW that FD.io<http://FD.io> project needs and that we can order any
time:

1. 28 NICs, 2p100 GbE from Nvidia / Mellanox - preferred:
   MCX613106A-VDAT, less preferred: MCX556A-EDAT, to cover the following
   testbeds:

   a. Performance 3-Node-ICX, 2 testbeds, 4 SUTs, 2 TGs
   b. Performance 2-Node-ICX, 4 testbeds, 4 SUTs, 4 TGs
   c. ICX TGs for other systems, 3 TGs
   d. 3-Node-Alt (Ampere Altra Arm N1), 1 testbed, 2 SUTs, 1 TG
   e. (exact breakdown in my email from 28 Jan 2022 in the thread below)

2. If we also want to add a MLX NIC for functional vpp_device test, that
   would be additional 2 MLX 2p100GbE NICs.

Things that we originally planned, but can't place orders as the HW is
not available yet:

3. TBC number of 2-socket Xeon SapphireRapids servers

   a. Intel Xeon processor SKUs are not available yet to us - expecting
  update any week now.
   b. Related SuperMicro SKUs are not available yet to us - expecting
  update any week now.

Hope this helps. Happy to answer any questions.

Cheers,
-Maciek

[1] 
https://ircbot.wl.linuxfoundation.org/meetings/fdio-meeting/2022/fd_io_tsc/fdio-meeting-fd_io_tsc.2022-06-09-15.00.html

On 5 Apr 2022, at 12:55, Maciek Konstantynowicz (mkonstan) 
mailto:mkons...@cisco.com>> wrote:

Super, thanks!

On 4 Apr 2022, at 20:22, Dave Wallace 
mailto:dwallac...@gmail.com>> wrote:

Hi Maciek,

I have added this information to the TSC Agenda [0].

Thanks,
-daw-
[0] https://wiki.fd.io/view/TSC#Agenda

On 4/4/2022 10:46 AM, Maciek Konstantynowicz (mkonstan) wrote:


Begin forwarded message:

From: mkonstan mailto:mkons...@cisco.com>>
Subject: Re: [tsc-private] [csit-dev] TRex - replacing CVL with MLX for 100GbE
Date: 3 March 2022 at 16:23:08 GMT
To: Ed Warnicke mailto:e...@cisco.com>>
Cc: "tsc-priv...@lists.fd.io<mailto:tsc-priv...@lists.fd.io>" 
mailto:tsc-priv...@lists.fd.io>>, Lijian Zhang 
mailto:lijian.zh...@arm.com>>

+Lijian

Hi,

Resending email from January, so it’s refreshed in our collective memory, as 
discussed on TSC call just now.

Number of 2p100GE MLX NICs needed for performance testing of Ampere Altra 
servers are listed under point 4 below.

Let me know if anything unclear and/or if any questions.

Cheers,
Maciek

On 28 Jan 2022, at 17:35, Maciek Konstantynowicz (mkonstan) via 
lists.fd.io<http://lists.fd.io/> 
mailto:mkonstan=cisco@lists.fd.io>> wrote:

Hi Ed, Trishan,

One correction regarding my last email from 25-Jan:-

For Intel Xeon Icelake testbeds, apart from just replacing E810s on TRex 
servers, we should also considder adding MLX 100GbE NICs for SUTs, so that 
FD.io<http://fd.io/> could benchmark MLX on latest Intel Xeon CPUs. Exactly as 
discussed on in our side conversation, Ed.

Here an updated calc with breakdown for Icelake (ICX) builds (the Cascadelake 
part stays as per previous email):

// Sorry to TL;DR, if you just want the number of NICs, scroll to the bottom of 
this message :)

(SUT, system under test, server running VPP+NICs under test)
(TG, traffic generator, server running TRex, needs link speeds matching SUTs')

1. 3-Node-ICX, 2 testbeds, 4 SUTs, 2 TGs
   - 4 SUT/VPP/dpdk servers
 - 4 ConnectX NIC, 1 per SUT - test ConnectX on SUT
   - 2 TG/TRex servers
 - 2 ConnectX NICs, 1 per TG - replace E810s and test E810 on SUT
 - 2 ConnectX NICs, 1 per TG - test ConnectX on SUT
 - 1 ConnectX NIC, 1 per testbed type - for TRex calibration
   - sub-total 9 NICs

2. 2-Node-ICX, 4 testbeds, 4 SUTs, 4 TGs
   - 4 SUT/VPP/dpdk servers
 - 4 ConnectX NIC, 1 per SUT - test ConnectX on SUT
   - 4 TG/TRex servers
 - 4 ConnectX NICs, 1 per TG - replace E810s and test E810 on SUT
 - 4 ConnectX NICs, 1 per TG - test ConnectX on SUT
 - 1 ConnectX NIC, 1 per testbed type - for TRex calibration
   - sub-total 13 NICs

3. ICX TGs for other systems, 3 TGs
   - 3 TG/TRex servers
 - 3 ConnectX NICs, 1 per TG - replace E810s and test ConnectX and other 
100GbE NICs on SUTs
 - 1 ConnectX NIC, 1 per testbed type - for TRex calibration
   - sub-total 4 NICs

4. 3-Node-Alt (Ampere Altra Arm N1), 1 testbed, 2 SUTs, 1 TG
   - 2 SUT/VPP/dpdk servers
 - 2 ConnectX NIC, 1 per SUT - test ConnectX on SUT
   - 1 TG/TRex server
 - will use one of the ICX TGs as listed in point 3.
   - sub-total 2 NICs

Total 28 NICs.

Hope this makes sense ...

Cheers,
Maciek

P.S. I'm on PTO now until 7-Feb, so email responses delayed.

On 25 Jan 2022, at 16:38, mkonstan 
mailto:mkons...@cisco.com>> wrote:

Hi Ed, Trishan,

Following from the last TSC call, here are the details about Nvidia Mellanox 
NICs that we are after for CSIT.

For existing Intel Xeon Cascadelake testbeds we have one option:

- MCX556A-EDAT NIC 2p100GbE - $1,195.00 - details in [2].
- need 4 NICs, plus 1 spare => 5 NICs

For the new Intel Xeon Icelake testbeds we have two op

[vpp-dev] CSIT and VPP 22.02 known issues status check

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi,

We have been discussing and working through a number of known issues since
VPP v22.02 and CSIT-2202 shipped.

Entering v22.06 rc2, so I thought it’s good time to take an
overall read of status, as jira is not always an accurate reflection.

Below my compilation based on CSIT-2202 report and listed jira
Tickets. 

Appreciate if folks could review and ack/nack/update the
status, and share their thoughts.

Cheers,
-Maciek



List of known issues in CSIT-2202 from
[CSIT-2202/vpp_performance_tests/csit_release_notes.txt]
(https://s3-docs.fd.io/csit/rls2202/report/_sources/vpp_performance_tests/csit_release_notes.rst.txt):

1. CSIT-1799 All NAT44-ED 16M CPS scale tests fail while setting NAT44 address 
range
  - https://jira.fd.io/browse/CSIT-1799

  - status: BACKLOG
  - error: Failed to set NAT44 address range on host 10.30.51.44 
(connections-per-second tests only)
  - symptom: VPP crash, The crash happens in the vlibmemory part of processing 
API commands.

  - dependencies:
+ VPP jira or patch ?

2. CSIT-1800 All Geneve L3 mode scale tests (1024 tunnels) are failing
  - https://jira.fd.io/browse/CSIT-1800

  - status: BACKLOG
  - error: Failed to add IP neighbor on interface geneve_tunnel258
  - symptom: VPP crash, Signal handler called from the main thread, from the 
code that adds new FIB entry. Looks like a VPP bug.

  - dependencies:
+ VPP jira or patch ?

3. CSIT-1801 9000B payload frames not forwarded over tunnels due to violating 
supported Max Frame Size (VxLAN, LISP, SRv6)
  - https://jira.fd.io/browse/CSIT-1801

  - status: BACKLOG
  - error: Minimal rate loss ratio 1.0 does not reach target 0.005
  - symptom: large frames are dropped

  - dependencies:
+ CSIT-1797 Standardize handling of max frame size
  * https://jira.fd.io/browse/CSIT-1797
  * status: BACKLOG

4. CSIT-1808 All tests with 9000B payload frames not forwarded over memif 
interfaces
  - https://jira.fd.io/browse/CSIT-1808

  - status: BACKLOG
  - error: Minimal rate loss ratio 1.0 does not reach target 0.005
  - symptom: large frames are dropped

5. CSIT-1809 All tests with 9000B payload frames not forwarded over vhostuser 
interfaces
  - https://jira.fd.io/browse/CSIT-1809

  - status: BACKLOG
  - error: Minimal rate loss ratio 1.0 does not reach target 0.005
  - symptom: large frames are dropped

6. CSIT-1802 AF-XDP - NDR tests failing from time to time
  - https://jira.fd.io/browse/CSIT-1802

  - status: BACKLOG
  - error: Minimal rate loss ratio does not reach target 0.0
  - symptom: af-xdp losing packets, loss ratio < 0.005 as PDR tests are passing
+ Small loss counts are possible even at low loads

7. CSIT-1803 3n-icx testbeds (Icelake): all IMIX aes128cbc-hmac512sha tests are 
failing due to excessive packet loss
  - https://jira.fd.io/browse/CSIT-1803

  - status: BACKLOG
  - error: Minimal rate loss ratio does not reach target 0.005
  - symptom: VPP on icx is dropping exactly the 570B component of IMIX (and 
forwarding both the 64B and the 1518B components)

8. CSIT-1804 3n-tsh testbed (Taishan): NDR tests failing from time to time
  - https://jira.fd.io/browse/CSIT-1804

  - status: BACKLOG
  - error: Minimal rate loss ratio does not reach target 0.0
  - symptom: At low loads, small number of frames are lost but only in one 
direction

9. CSIT-1805 3n-skx testbeds (Skylake): all hoststack quic vppecho scale tests 
are failing
  - https://jira.fd.io/browse/CSIT-1805

  - status: OPEN / FIXED?
  - error: SSHTimeout
  - symptom: Failed to run 'cli_inband show logging'

10. CSIT-1810 DPDK - performance regression with DPDK driver when Max Frame 
Size is set to less than 2023
   - https://jira.fd.io/browse/CSIT-1810

   - status: DONE
   - dependencies:
 + VPP-1876 Worse performance with DPDK driver when MTU is set to 2022 or 
less
   * https://jira.fd.io/browse/VPP-1876
   * status: DONE

11. CSIT-1811 All 9000B NAT44DET 64k 1m scale tests fail due to bps rate set to 
high on TRex
   - https://jira.fd.io/browse/CSIT-1811

   - status: DONE

12. CSIT-1812 All IMIX NAT44DET 4m 16m scale tests fail due to not creating 
required session count
   - https://jira.fd.io/browse/CSIT-1812

   - status: BACKLOG
   - error: Not all DET44 sessions have been established
   - symptom: `4m:   2747500 != 4128768`, `16m: 9747501 != 16515072`


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21540): https://lists.fd.io/g/vpp-dev/message/21540
Mute This Topic: https://lists.fd.io/mt/91751011/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] New CSIT dashboard with VPP and DPDK performance data

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi,

Following from Tibor's demo of the new CSIT dashboard on the VPP call
last Tuesday, here the links to what was presented.

Note that the backend and frontend apps are still being tuned, so please
treat this as an engineering preview. Glitches and disruption are
likely to occur :)

- Current landing page: http://csit.fd.io/
 + Performance trending - dynamic dashboard with user defined views.
 + Job statistics - csit perf jobs execution data, richer then jenkins.

- Functionality highlights
 + User selectable inputs
   * DUT, Infra, Area, Test
   * Frame size, Number of cores, Test type
 + Graphs with detected anomaly highlights
   * Throughput graphs for MRR, PDR and NDR test types
   * Latency graphs for PDR test types
   * Data point click for off-canvas display of details
 + Saving options
   * Custom view encoded in the URL
   * Custom view test data in csv file

- Some example views with recent anomalies
 + avf ip4 2n-clx - regression on 04/18 [1]
 + avf ip4 2n-clx 2n-icx - regression 04/18 [2]
 + added dpdk 2n-clx ip4 - progression 03/15 [3]
 + dpdk ip4 base 1c 2c 4c 3n-alt - multi-core speed-up [4]
 + dpdk ip6 3n-alt - regression 05/17 [5]

Please use csit-...@lists.fd.io for all feedback and comments.

Cheers,
Maciek


[1] avf ip4 2n-clx, 
http://csit.fd.io/trending/#eNrVkk0KwjAQhU9TNzKQjq1dufDnHhLTUYtNDUks1dObSmEs6E4kXWRC-B4zk8dz_mpp76heJfkmKTYJFlUZSrJYz8PVGgPYgKo7wPxEaNKua4tUgGyPUJkMltkBUgXkz_0rHKdkTSjEBbS1fSPc9o3Kmx91ZWLOdybfZrFcWpKsDwMZeXJvQz6sxNKjlZpc9SDWh58wV8EVRqkaD_F380aHfxa7l-KnFtqmjNDGfq1pWRlfGC9TMzDGKE4viTo2D_XE7IswhvofKcx3M-el9SsUiCAQEOdCBNlQZtSUA8sBsxF7AkMkH_s

[2] avf ip4 2n-clx 2n-icx, 
http://csit.fd.io/trending/#eNrVkEEOgjAQRU-DGzNJGUFWLFTuYWoZhQjYtJWgp7cYkpFEd0ZhQUnzfue3z7qLob2lKg3ibZBsA0zK3C_BarP0v1ZrwAZU1QHGJ0Iddl2bhAJke4RSR7CODhAqIFf0O_9ZJStCIc5gmhxqY_phuOuH5Vc3msxEFzcmn_o4Lg1JzvtSRo7sS8mHa3H8aGRNtrwTn_EvYq68HUahGhe5m36hw1uT7Jn4nsppmpyfyHqKHut_ayzVLzW-bZuLxjhbWCeNS1EggkBAXArhY8OyoCYfWAwYjdgD2tl11g

[3] added dpdk 2n-clx ip4, 
http://csit.fd.io/trending/#eNrVkk0OgjAQhU-DGzJJGUFWLkTvYWo7KuHHpq0EPb3FkIws3BkDi07TfK_zOi91_mbp6KjeRlkR5UWEealDida7OGydMYAtqLoHzC6EJun7Lk8EyO4MpUlhk54gUUD-OpzCckrWhEJUYFsNjbVDM9wPzfTdTzozMdcHk29-LJeWJOuDKSNP7sPky7NYfrayIVc-ie-EiZirkA6jRE2N_MN80HHW_PBW_C7KeSa5vCCbOebYzC9GbXT1zw85-C3mR2aHlfPS-i0KRBAIiLEQQTaWFbV6ZBlgOmEv6bp3BA

[4] dpdk ip4 base 1c 2c 4c 3n-alt, 
http://csit.fd.io/trending/#eNrdkMEOgjAQRL8GL2STshQ5eRD5D1PpKkTApq1G_HqLIVk5mBgTLxy6TfMmM9tx_mJp76jdRFkR5UWEeaPDiNJtHK6bMZD2oFoPUpwITXJv80SANvoMjZGwlgdIKiBfj69wDsoRdNaOHrgbPfTVzwyZmHpg8imG5cqSYn3IYuTJvYXMt2HV0aqOXPMglob9mVehC0ZJNff3g3mj0xfz8qX4rThcZnH49-LkMouTXxaXlSvnlfUbFIggEBBjIYJsGivq9cQyQDljTy_DDDc

[5] dpdk ip6 3n-alt, 
http://csit.fd.io/trending/#eNrVkM0OgjAQhJ8GL2aTsoL1wsGf9zCVrkJEbdpqxKe3GOLSgzdN8NBtmm8ysx3nL5a2jpoiyVeJXCUoax1GMltOw3UzBmZnUI2HTBwITXpvZCpAG32E2sxBLnaQlkC-6l7h7JQjOFnbeeC689BXHxkyMVXL5FMMy5UlxfqQxciTG4TE27Bqb9WJXP0glob9mZehC0ZpGfv71gxo_0W5eSm-UJwrVUMoxHE87b1X-q8Kx9fgzwvMNxPnlfUFCkQQCIhTIYKsHxM6657lgFnEnq0vEbk
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21469): https://lists.fd.io/g/vpp-dev/message/21469
Mute This Topic: https://lists.fd.io/mt/91352683/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP call topic - csit.fd.io perf dashboard preview

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi Damjan,

For the VPP call today, if possible, could we get 10-15min for
CSIT team to do a demo of the new csit.fd.io perf dashboard?

Cheers,
Maciek

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21439): https://lists.fd.io/g/vpp-dev/message/21439
Mute This Topic: https://lists.fd.io/mt/91309098/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT topics for VPP meeting today

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi,

We would like to spend 5-10min doing the CSIT-2202 vpp rls notes review, 
specifically recorder failures and regressions.
Vratko and me will lead that.

If there is no time on the call agenda today, no sweat, we can do it on the 
next call.

Cheers,
Maciek
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21239): https://lists.fd.io/g/vpp-dev/message/21239
Mute This Topic: https://lists.fd.io/mt/90418453/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] topic proposal for today's vpp community call

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi,

If possible, in today’s meeting, we would like to cover highlights of vpp 
performance regressions and test failures based on v22.02 rc2 testing in CSIT. 
5-10 min will suffice to cover the highlights.

Vratko will be sending current summary in email to vpp-dev and csit-dev before 
the meeting.

Apologies for sending this late, will understand if there is no time to cover 
this :)

Cheers,
Maciek
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20887): https://lists.fd.io/g/vpp-dev/message/20887
Mute This Topic: https://lists.fd.io/mt/89318418/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Thu 13-Jan FD.io CSIT talk at LFN Developer & Testing Forum

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Here pointers to materials and recordings from the session last week - Enjoy:


CSIT Talk: Performance Benchmarking, Anomaly Detection, Results Tracking

  Session wiki page [1].

  Slides lfn-ddts21-fdio-csit.pdf [2].

  Recording lfn-ddts21-fdio-csit.mp4 [3].

  Topics - with timestamps in recording:

FD.io Overview 
  VPP and CSIT core projects - 1m 50s
FD.io CSIT
  Daily Trending and Release Reports - 11m 35s
  Anomaly Detection - 24m 30s
  Data Processing Infrastructure - 33m 20s
Wrap-up - 39m 50s
Q - 42m 0s
Backup slide - jumpavg Compression Algorithm - 56m 50s
End - 1h 04m 0s

[1] https://wiki.lfnetworking.org/pages/viewpage.action?pageId=65538382
[2] 
https://wiki.lfnetworking.org/download/attachments/65538382/lfn-ddts21-fdio-csit.pdf?version=1=1642187365000=v2
[3] 
https://wiki.lfnetworking.org/download/attachments/65538382/lfn-ddts21-fdio-csit.mp4?version=1=1642175623000=v2


> On 11 Jan 2022, at 15:58, mkonstan  wrote:
> 
> Dear All,
> 
> Some of us from CSIT will be giving a talk at this week LFN Developer &
> Testing Forum (virtual event via Zoom, and it’s free to attend).
> 
> Title: FD.io CSIT Performance Benchmarking, Anomaly Detection, Results 
> Tracking.
> 
> Time: Thursday 13th January 2021 13:00-14:00 GMT (London)
> 
> Abstract: 
> 
> In this talk we introduce FD.io CSIT core sub-project and give an
> overview of the ways its Presentation-Analytics-Layer component
> presents test results. We will focus on performance results and anomaly
> detection, as a way to distinguish (probably) real performance changes
> from the noisy data. And provide a sneak peak into the way test results
> data is processed. Will use performance data from CI/CD automated tests
> executed on Intel Xeon(skx, clx, icx), Atom, Arm servers running in
> labs hosted by LFN FD.io. If you are interested in network performance
> testing in general, or FD.io project specifically, this talk is for
> you.
> 
> For FD.io CSIT session details see: 
> 
>https://wiki.lfnetworking.org/pages/viewpage.action?pageId=65538382
> 
> Event details and how to register: 
> 
>https://wiki.lfnetworking.org/pages/viewpage.action?pageId=53609047
> 
> Detailed schedule (make sure to select your timezone): 
> 
>https://teamup.com/ksgw6qzqcmg9zbzsfq?date=2022-01-10=md4
> 
> Cheers,
> Maciek
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20768): https://lists.fd.io/g/vpp-dev/message/20768
Mute This Topic: https://lists.fd.io/mt/88351902/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] Thu 13-Jan FD.io CSIT talk at LFN Developer & Testing Forum

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Zoom link to the session per teamup link below: 
https://zoom.us/j/96267999834?pwd=UnBNM2NveXgyT3BpdkNSenV2VjJjQT09

We will post the materials on the event wiki link listed below and in fd.io 
wiki, will send links.

> On 11 Jan 2022, at 15:58, Maciek Konstantynowicz (mkonstan) via lists.fd.io 
>  wrote:
> 
> Dear All,
> 
> Some of us from CSIT will be giving a talk at this week LFN Developer &
> Testing Forum (virtual event via Zoom, and it’s free to attend).
> 
> Title: FD.io CSIT Performance Benchmarking, Anomaly Detection, Results 
> Tracking.
> 
> Time: Thursday 13th January 2021 13:00-14:00 GMT (London)
> 
> Abstract: 
> 
> In this talk we introduce FD.io CSIT core sub-project and give an
> overview of the ways its Presentation-Analytics-Layer component
> presents test results. We will focus on performance results and anomaly
> detection, as a way to distinguish (probably) real performance changes
> from the noisy data. And provide a sneak peak into the way test results
> data is processed. Will use performance data from CI/CD automated tests
> executed on Intel Xeon(skx, clx, icx), Atom, Arm servers running in
> labs hosted by LFN FD.io. If you are interested in network performance
> testing in general, or FD.io project specifically, this talk is for
> you.
> 
> For FD.io CSIT session details see: 
> 
>https://wiki.lfnetworking.org/pages/viewpage.action?pageId=65538382
> 
> Event details and how to register: 
> 
>https://wiki.lfnetworking.org/pages/viewpage.action?pageId=53609047
> 
> Detailed schedule (make sure to select your timezone): 
> 
>https://teamup.com/ksgw6qzqcmg9zbzsfq?date=2022-01-10=md4
> 
> Cheers,
> Maciek
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20707): https://lists.fd.io/g/vpp-dev/message/20707
Mute This Topic: https://lists.fd.io/mt/88395628/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Thu 13-Jan FD.io CSIT talk at LFN Developer & Testing Forum

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Dear All,

Some of us from CSIT will be giving a talk at this week LFN Developer &
Testing Forum (virtual event via Zoom, and it’s free to attend).

Title: FD.io CSIT Performance Benchmarking, Anomaly Detection, Results Tracking.

Time: Thursday 13th January 2021 13:00-14:00 GMT (London)

Abstract: 

In this talk we introduce FD.io CSIT core sub-project and give an
overview of the ways its Presentation-Analytics-Layer component
presents test results. We will focus on performance results and anomaly
detection, as a way to distinguish (probably) real performance changes
from the noisy data. And provide a sneak peak into the way test results
data is processed. Will use performance data from CI/CD automated tests
executed on Intel Xeon(skx, clx, icx), Atom, Arm servers running in
labs hosted by LFN FD.io. If you are interested in network performance
testing in general, or FD.io project specifically, this talk is for
you.

For FD.io CSIT session details see: 

https://wiki.lfnetworking.org/pages/viewpage.action?pageId=65538382

Event details and how to register: 

https://wiki.lfnetworking.org/pages/viewpage.action?pageId=53609047

Detailed schedule (make sure to select your timezone): 

https://teamup.com/ksgw6qzqcmg9zbzsfq?date=2022-01-10=md4

Cheers,
Maciek


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20697): https://lists.fd.io/g/vpp-dev/message/20697
Mute This Topic: https://lists.fd.io/mt/88351902/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] FD.io CSIT-2110 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Dear Ameen, pls see inline.

On 11 Nov 2021, at 20:19, Ameen Al-Azzawi 
mailto:ameen.azz...@gmail.com>> wrote:

Dear Maciek,

I didn't mean to be rude, but you know how it feels when you see a community of 
professionals for specific software and no one could help.

Understand.

I know that no one is obliged to help me, I was simply expecting more out of it.

My read is that folks are simply very busy, and number of people versed in 
ds-lite and other softwire technologies that VPP implements aplenty of, is not 
that large. So pls bear with this community, and keep trying. Someone wise once 
said "Energy and (constructive) persistence conquer all things”, especially 
when done with conviction of being right ;)

Cheers,
Maciek


Regards
Ameen

On Thu, Nov 11, 2021 at 4:39 PM Maciek Konstantynowicz (mkonstan) 
mailto:mkons...@cisco.com>> wrote:
Dear Ameen,

What specific statement in the email do you disagree with?

This is not a research paper, it is FD.io<http://fd.io/> CSIT release report 
capturing benchmarking and functional test data for v21.10 release.

Regarding your comment about "VPP and the system doesn't function”, I checked 
your statement about:

"I have asked here a million times about configuring ds-lite, no one helped.”

And found out that you have sent few emails (between 26-Sep and 28-Oct) to 
vpp-dev with questions about running ds-lite on vpp, and the last email was not 
answered.
I suggest you resend the email, and I am confident that FD.io<http://fd.io/> 
dev community will provide you with support.

Cheers,
Maciek


On 11 Nov 2021, at 14:22, Ameen Al-Azzawi 
mailto:ameen.azz...@gmail.com>> wrote:

I would totally disagree.
It is disappointing when you base your research paper on VPP and the system 
doesn't function.
I have asked here a million times about configuring ds-lite, no one helped.

On Thu, Nov 11, 2021 at 3:19 PM Maciek Konstantynowicz (mkonstan) via 
lists.fd.io<http://lists.fd.io/> 
mailto:cisco@lists.fd.io>> wrote:
Big thanks to FD.io<http://fd.io/> community contributors and supporters that 
enabled this data-rich publication.
Great job everyone!

Cheers,
Maciek

On 10 Nov 2021, at 18:12, Tibor Frank via lists.fd.io<http://lists.fd.io/> 
mailto:tifrank=cisco@lists.fd.io>> wrote:

Hi All,

FD.io<http://fd.io/> CSIT-2110 report is now available on FD.io<http://fd.io/> 
docs site:

https://s3-docs.fd.io/csit/rls2110/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2110 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to 
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>.

Tibor


CSIT-2110 Release Summary
-

BENCHMARK TESTS

- Intel Xeon Ice Lake: Added test data for these platforms. Current CSIT-2110
  report data for Intel Xeon Ice Lake comes from an external source (Intel
  labs running CSIT code on “8360Y D Stepping” and “6338N” processors).

- MLRsearch improvements: Added support for multiple packet throughput rates
  in a single search, each rate is associated with a distinct Packet Loss Ratio
  (PLR) criterion. Previously only Non Drop Rate (NDR) (PLR=0) and single
  Partial Drop Rate (PDR) (PLR<0.5%) were supported. Implemented number of
  optimizations improving rate discovery efficiency.

- TREX performance tests: Added initial tests for testing latency between 2
  ports on nic on the TRex. Added tests: IP4Base, IP4scale2m, IP6Base,
  IP6scale2m, L2bscale1mmaclrn.

TEST FRAMEWORK

- CSIT test environment version has been updated to ver. 8: Intel NIC 700/800
  series firmware upgrade based on DPDK compatibility matrix.

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
  environment.

Pointers to CSIT-2110 Report sections
-

1. FD.io<http://fd.io/> CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.10 vs. v21.06[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 21.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.08 vs. 21.02[11]
12. TRex 2.88 apps release notes[12]
13. TRex 64B throughput graphs  [13]
14. TRex latency under load [14]

Functional device tests (VPP_Device) are also included in the report.

[1] https://s3-docs.fd.io/csit/rls2110/report/introduction/methodology.html
[2] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://s3-docs.fd.io/csit/rls2110/rep

Re: [vpp-dev] [csit-dev] FD.io CSIT-2110 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Dear Ameen,

What specific statement in the email do you disagree with?

This is not a research paper, it is FD.io<http://FD.io> CSIT release report 
capturing benchmarking and functional test data for v21.10 release.

Regarding your comment about "VPP and the system doesn't function”, I checked 
your statement about:

"I have asked here a million times about configuring ds-lite, no one helped.”

And found out that you have sent few emails (between 26-Sep and 28-Oct) to 
vpp-dev with questions about running ds-lite on vpp, and the last email was not 
answered.
I suggest you resend the email, and I am confident that FD.io<http://FD.io> dev 
community will provide you with support.

Cheers,
Maciek


On 11 Nov 2021, at 14:22, Ameen Al-Azzawi 
mailto:ameen.azz...@gmail.com>> wrote:

I would totally disagree.
It is disappointing when you base your research paper on VPP and the system 
doesn't function.
I have asked here a million times about configuring ds-lite, no one helped.

On Thu, Nov 11, 2021 at 3:19 PM Maciek Konstantynowicz (mkonstan) via 
lists.fd.io<http://lists.fd.io/> 
mailto:cisco@lists.fd.io>> wrote:
Big thanks to FD.io<http://fd.io/> community contributors and supporters that 
enabled this data-rich publication.
Great job everyone!

Cheers,
Maciek

On 10 Nov 2021, at 18:12, Tibor Frank via lists.fd.io<http://lists.fd.io/> 
mailto:tifrank=cisco@lists.fd.io>> wrote:

Hi All,

FD.io<http://fd.io/> CSIT-2110 report is now available on FD.io<http://fd.io/> 
docs site:

https://s3-docs.fd.io/csit/rls2110/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2110 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to 
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>.

Tibor


CSIT-2110 Release Summary
-

BENCHMARK TESTS

- Intel Xeon Ice Lake: Added test data for these platforms. Current CSIT-2110
  report data for Intel Xeon Ice Lake comes from an external source (Intel
  labs running CSIT code on “8360Y D Stepping” and “6338N” processors).

- MLRsearch improvements: Added support for multiple packet throughput rates
  in a single search, each rate is associated with a distinct Packet Loss Ratio
  (PLR) criterion. Previously only Non Drop Rate (NDR) (PLR=0) and single
  Partial Drop Rate (PDR) (PLR<0.5%) were supported. Implemented number of
  optimizations improving rate discovery efficiency.

- TREX performance tests: Added initial tests for testing latency between 2
  ports on nic on the TRex. Added tests: IP4Base, IP4scale2m, IP6Base,
  IP6scale2m, L2bscale1mmaclrn.

TEST FRAMEWORK

- CSIT test environment version has been updated to ver. 8: Intel NIC 700/800
  series firmware upgrade based on DPDK compatibility matrix.

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
  environment.

Pointers to CSIT-2110 Report sections
-

1. FD.io<http://fd.io/> CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.10 vs. v21.06[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 21.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.08 vs. 21.02[11]
12. TRex 2.88 apps release notes[12]
13. TRex 64B throughput graphs  [13]
14. TRex latency under load [14]

Functional device tests (VPP_Device) are also included in the report.

[1] https://s3-docs.fd.io/csit/rls2110/report/introduction/methodology.html
[2] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_latency/index.html
[6] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://s3-docs.fd.io/csit/rls2110/report/detailed_test_results/vpp_performance_results/index.html
[8] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/csit_release_notes.html
[9] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/packet_latency/index.html
[11] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html
[12] 
https://s3-docs.fd.io/csit/rls2110/report/trex_performance_tests/csit_

Re: [vpp-dev] [csit-dev] FD.io CSIT-2110 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi Stanislav,

You are welcome.

Thanks for spotting, and you are right, the af_xdp driver numbers did a nose 
dive indeed.
It’s visible on xeon skylake per your link references, and also on:
- intel xeon cascadelake [2], [3],
- arm thunderx2 [4].

Comparison tables [5], [6] provide a proof that the drop is due to CSIT test 
environment change from v7 to v8 [7], and unlikely due to VPP code.
We need to investigate.

Hope this makes sense.

Cheers,
Maciek

[2] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-clx-xxv710.html#b-2t1c-ip4routing-base-scale-af-xdp
[3] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-clx-xxv710.html#b-2t1c-ip4routing-base-scale-af-xdp
[4] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-tx2-xl710.html#b-1t1c-ip4routing-base-scale-af-xdp
[5] search for “af-xdp” in 
https://s3-docs.fd.io/csit/rls2110/report/_static/vpp/performance-changes-2n-skx-2t1c-pdr.txt
[6] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/test_environment.html#environment-versioning

On 11 Nov 2021, at 14:47, Stanislav Zaikin 
mailto:zsta...@gmail.com>> wrote:

Thanks for the report!
Out of curiosity, what happened to AF_XDP? PPS rate was much bigger on r21.06.

[0] - 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-skx-x710.html
[1] - 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-skx-x710.html

On Thu, 11 Nov 2021 at 15:19, Maciek Konstantynowicz (mkonstan) via 
lists.fd.io<http://lists.fd.io/> 
mailto:cisco@lists.fd.io>> wrote:
Big thanks to FD.io<http://fd.io/> community contributors and supporters that 
enabled this data-rich publication.
Great job everyone!

Cheers,
Maciek

On 10 Nov 2021, at 18:12, Tibor Frank via lists.fd.io<http://lists.fd.io/> 
mailto:tifrank=cisco@lists.fd.io>> wrote:

Hi All,

FD.io<http://fd.io/> CSIT-2110 report is now available on FD.io<http://fd.io/> 
docs site:

https://s3-docs.fd.io/csit/rls2110/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2110 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to 
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>.

Tibor


CSIT-2110 Release Summary
-

BENCHMARK TESTS

- Intel Xeon Ice Lake: Added test data for these platforms. Current CSIT-2110
  report data for Intel Xeon Ice Lake comes from an external source (Intel
  labs running CSIT code on “8360Y D Stepping” and “6338N” processors).

- MLRsearch improvements: Added support for multiple packet throughput rates
  in a single search, each rate is associated with a distinct Packet Loss Ratio
  (PLR) criterion. Previously only Non Drop Rate (NDR) (PLR=0) and single
  Partial Drop Rate (PDR) (PLR<0.5%) were supported. Implemented number of
  optimizations improving rate discovery efficiency.

- TREX performance tests: Added initial tests for testing latency between 2
  ports on nic on the TRex. Added tests: IP4Base, IP4scale2m, IP6Base,
  IP6scale2m, L2bscale1mmaclrn.

TEST FRAMEWORK

- CSIT test environment version has been updated to ver. 8: Intel NIC 700/800
  series firmware upgrade based on DPDK compatibility matrix.

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
  environment.

Pointers to CSIT-2110 Report sections
-

1. FD.io<http://fd.io/> CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.10 vs. v21.06[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 21.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.08 vs. 21.02[11]
12. TRex 2.88 apps release notes[12]
13. TRex 64B throughput graphs  [13]
14. TRex latency under load [14]

Functional device tests (VPP_Device) are also included in the report.

[1] https://s3-docs.fd.io/csit/rls2110/report/introduction/methodology.html
[2] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_latency/index.html
[6] 
https://s3-docs.fd.io/csit/rls2110/report

Re: [vpp-dev] [csit-dev] FD.io CSIT-2110 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Big thanks to FD.io community contributors and supporters that 
enabled this data-rich publication.
Great job everyone!

Cheers,
Maciek

On 10 Nov 2021, at 18:12, Tibor Frank via lists.fd.io 
mailto:tifrank=cisco@lists.fd.io>> wrote:

Hi All,

FD.io CSIT-2110 report is now available on FD.io 
docs site:

https://s3-docs.fd.io/csit/rls2110/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2110 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to 
csit-...@lists.fd.io.

Tibor


CSIT-2110 Release Summary
-

BENCHMARK TESTS

- Intel Xeon Ice Lake: Added test data for these platforms. Current CSIT-2110
  report data for Intel Xeon Ice Lake comes from an external source (Intel
  labs running CSIT code on “8360Y D Stepping” and “6338N” processors).

- MLRsearch improvements: Added support for multiple packet throughput rates
  in a single search, each rate is associated with a distinct Packet Loss Ratio
  (PLR) criterion. Previously only Non Drop Rate (NDR) (PLR=0) and single
  Partial Drop Rate (PDR) (PLR<0.5%) were supported. Implemented number of
  optimizations improving rate discovery efficiency.

- TREX performance tests: Added initial tests for testing latency between 2
  ports on nic on the TRex. Added tests: IP4Base, IP4scale2m, IP6Base,
  IP6scale2m, L2bscale1mmaclrn.

TEST FRAMEWORK

- CSIT test environment version has been updated to ver. 8: Intel NIC 700/800
  series firmware upgrade based on DPDK compatibility matrix.

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
  environment.

Pointers to CSIT-2110 Report sections
-

1. FD.io CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.10 vs. v21.06[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 21.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.08 vs. 21.02[11]
12. TRex 2.88 apps release notes[12]
13. TRex 64B throughput graphs  [13]
14. TRex latency under load [14]

Functional device tests (VPP_Device) are also included in the report.

[1] https://s3-docs.fd.io/csit/rls2110/report/introduction/methodology.html
[2] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/packet_latency/index.html
[6] 
https://s3-docs.fd.io/csit/rls2110/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://s3-docs.fd.io/csit/rls2110/report/detailed_test_results/vpp_performance_results/index.html
[8] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/csit_release_notes.html
[9] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/packet_latency/index.html
[11] 
https://s3-docs.fd.io/csit/rls2110/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html
[12] 
https://s3-docs.fd.io/csit/rls2110/report/trex_performance_tests/csit_release_notes.html
[13] 
https://s3-docs.fd.io/csit/rls2110/report/trex_performance_tests/packet_throughput_graphs/index.html
[14] 
https://s3-docs.fd.io/csit/rls2110/report/trex_performance_tests/packet_latency/index.html






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20479): https://lists.fd.io/g/vpp-dev/message/20479
Mute This Topic: https://lists.fd.io/mt/86982560/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] FD.io CSIT-2106 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Caveat note :)

Number of the latency graphs in report show excessive numbers O(sec)
across frame sizes w/o and with load for both VPP and DPDK. 

Few of us in CSIT are working on the resolution, stay tuned. 
For now please ignore those numbers.

Cheers,
Maciek

> On 15 Jul 2021, at 13:03, Maciek Konstantynowicz (mkonstan) 
>  wrote:
> 
> Hi All,
> 
> FD.io CSIT-2106 report is now available on FD.io docs site:
> 
>   https://docs.fd.io/csit/rls2106/report/
> 
> Another successful release! Many thanks to all contributors in CSIT and
> VPP communities for making it happen.
> 
> See below for CSIT-2106 release summary and pointers to specific
> sections in the report.
> 
> Welcome all comments, best by email to csit-...@lists.fd.io.
> 
> Cheers,
> Maciek (CSIT PTL)
> 
> 
> CSIT-2106 Release Summary
> -
> 
> BENCHMARK TESTS
> 
> - AF_XDP: Added af_xdp driver support for all test cases. Test results
>  will be added in sub-sequent CSIT-2106 report.
> 
> - GTPU tunnel: Added GTPU HW Offload IPv4 routing tests.
> 
> - Intel Xeon Ice Lake: Added initial test data for these platforms.
>  Current CSIT-2106 report data for Intel Xeon Ice Lake comes from an
>  external source (Intel labs running CSIT code on
>  "8360Y D Stepping" and "6338N" processors).
> 
> - CSIT in AWS environment: Added CSIT support for AWS c5n instances
>  environment. Test results will be added in sub-sequent CSIT-2106
>  report.
> 
> - MLRsearch improvements: Added support for multiple packet throughput
>  rates in a single search, each rate is associated with a distinct
>  Packet Loss Ratio (PLR) criterion. Implemented number of
>  optimizations improving rate discovery efficiency.
> 
> TEST FRAMEWORK
> 
> - Telemetry retouch: Refactored telemetry retrieval from DUTs and SUTs.
>  Included VPP perfmon plugin telemetry with all perfmon bundles
>  available in VPP release.
> 
> - Upgrade to Ubuntu 20.04 LTS: Re-installed base operating system to
>  Ubuntu 20.04.2 LTS. Upgrade included also baseline Docker containers
>  used for spawning topology.
> 
> Pointers to CSIT-2106 Report sections
> -
> 
> 1. FD.io CSIT test methodology  [1]
> 2. VPP release notes[2]
> 3. VPP 64B/IMIX throughput graphs   [3]
> 4. VPP throughput speedup multi-core[4]
> 5. VPP latency under load   [5]
> 6. VPP comparisons v21.06 vs. v21.01[6]
> 7. VPP performance all pkt sizes & NICs [7]
> 8. DPDK 20.08 apps release notes[8]
> 9. DPDK 64B throughput graphs   [9]
> 10. DPDK latency under load [10]
> 11. DPDK comparisons 21.02 vs. 20.11[11]
> 
> Functional device tests (VPP_Device) are also included in the report.
> 
> [1] https://docs.fd.io/csit/rls2106/report/introduction/methodology.html
> [2] 
> https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/csit_release_notes.html
> [3] 
> https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/index.html
> [4] 
> https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
> [5] 
> https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_latency/index.html
> [6] 
> https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
> [7] 
> https://docs.fd.io/csit/rls2106/report/detailed_test_results/vpp_performance_results/index.html
> [8] 
> https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/csit_release_notes.html
> [9] 
> https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_throughput_graphs/index.html
> [10] 
> https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_latency/index.html
> [11] 
> https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19784): https://lists.fd.io/g/vpp-dev/message/19784
Mute This Topic: https://lists.fd.io/mt/84223353/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io CSIT-2106 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi All,

FD.io CSIT-2106 report is now available on FD.io docs site:

   https://docs.fd.io/csit/rls2106/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2106 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
Maciek (CSIT PTL)


CSIT-2106 Release Summary
-

BENCHMARK TESTS

- AF_XDP: Added af_xdp driver support for all test cases. Test results
  will be added in sub-sequent CSIT-2106 report.

- GTPU tunnel: Added GTPU HW Offload IPv4 routing tests.

- Intel Xeon Ice Lake: Added initial test data for these platforms.
  Current CSIT-2106 report data for Intel Xeon Ice Lake comes from an
  external source (Intel labs running CSIT code on
  "8360Y D Stepping" and "6338N" processors).

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
  environment. Test results will be added in sub-sequent CSIT-2106
  report.

- MLRsearch improvements: Added support for multiple packet throughput
  rates in a single search, each rate is associated with a distinct
  Packet Loss Ratio (PLR) criterion. Implemented number of
  optimizations improving rate discovery efficiency.

TEST FRAMEWORK

- Telemetry retouch: Refactored telemetry retrieval from DUTs and SUTs.
  Included VPP perfmon plugin telemetry with all perfmon bundles
  available in VPP release.

- Upgrade to Ubuntu 20.04 LTS: Re-installed base operating system to
  Ubuntu 20.04.2 LTS. Upgrade included also baseline Docker containers
  used for spawning topology.

Pointers to CSIT-2106 Report sections
-

1. FD.io CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.06 vs. v21.01[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 20.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.02 vs. 20.11[11]

Functional device tests (VPP_Device) are also included in the report.

[1] https://docs.fd.io/csit/rls2106/report/introduction/methodology.html
[2] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_latency/index.html
[6] 
https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://docs.fd.io/csit/rls2106/report/detailed_test_results/vpp_performance_results/index.html
[8] 
https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/csit_release_notes.html
[9] 
https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] 
https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_latency/index.html
[11] 
https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19783): https://lists.fd.io/g/vpp-dev/message/19783
Mute This Topic: https://lists.fd.io/mt/84223353/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP: A Terabit Secure Network Data-plane

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi, Arm tests are in CSIT reports e.g.

Taishan:
- ip4 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/packet_throughput_graphs/ip4-3n-tsh-x520.html
- ipsec 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/packet_throughput_graphs/ipsec-3n-tsh-x520.html

ThunderX2:
- ip4 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-tx2-xl710.html

But these are older Arms. No N1/Ampere in LFN FD.io labs yet … been asking for 
a while.

Cheers,
Maciek

> On 7 Apr 2021, at 21:43, hemant via lists.fd.io 
>  wrote:
> 
> Thanks for sharing.  
> 
> Did you test on ARM to see what the performance is?
> 
> Hemant
> 
> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Maciek
> Konstantynowicz (mkonstan) via lists.fd.io
> Sent: Wednesday, April 07, 2021 10:02 AM
> To: csit-dev ; vpp-dev 
> Subject: [vpp-dev] VPP: A Terabit Secure Network Data-plane
> 
> https://www.youtube.com/watch?v=ipQQmjzE_g0
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19134): https://lists.fd.io/g/vpp-dev/message/19134
Mute This Topic: https://lists.fd.io/mt/81916291/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP: A Terabit Secure Network Data-plane

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

https://www.youtube.com/watch?v=ipQQmjzE_g0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19124): https://lists.fd.io/g/vpp-dev/message/19124
Mute This Topic: https://lists.fd.io/mt/81916291/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io CSIT-2101 Release Report is published

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi All,

FD.io CSIT-2101 report is available on FD.io docs site:

   https://docs.fd.io/csit/rls2101/report/

Many thanks to all contributors in CSIT and VPP communities!

See below for CSIT-2101 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
Maciek (CSIT PTL)


CSIT-2101 Release Summary
-

NEW BENCHMARK TESTS

- GENEVE tests: Added VPP performance tests for GENEVE tunnels.

- NAT44 tests: Added new test type, pure throughput tests, measuring
  bi-directional forwarding performance after all stateful sessions are
  created.

- Jumbo for IPsec: Test cases with 9000 byte frames are re-enabled in
  ipsec suites.

- IPv4, IPv6 scale: Added randomized traffic profiles for IPv6 scale
  tests, improved repeatability.

- Arm 2n-tx2 testbed: New physical testbed type added with VPP and DPDK
  performance data added to CSIT trending and this report.

TEST FRAMEWORK

- TRex ASTF: Improved capability to run TRex in advanced stateful mode.

- CSIT PAPI support: Due to issues with PAPI performance, VAT is still
  used in CSIT for all VPP scale tests.

PRESENTATION

- Packet Latency Graphs: Updated with logarithmic X-Axis providing more
  granular view of per packet latency.


Pointers to CSIT-2101 Report sections
-

1. FD.io CSIT test methodology  [1]
2. VPP release notes[2]
3. VPP 64B/IMIX throughput graphs   [3]
4. VPP throughput speedup multi-core[4]
5. VPP latency under load   [5]
6. VPP comparisons v21.01 vs. v20.09[6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 20.08 apps release notes[8]
9. DPDK 64B throughput graphs   [9]
10. DPDK latency under load [10]
11. DPDK comparisons 20.11 vs. 20.08[11]

Functional device tests (VPP_Device) are also included in the report.

[1] https://docs.fd.io/csit/rls2101/report/introduction/methodology.html
[2] 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/csit_release_notes.html
[3] 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/packet_latency/index.html
[6] 
https://docs.fd.io/csit/rls2101/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://docs.fd.io/csit/rls2101/report/detailed_test_results/vpp_performance_results/index.html
[8] 
https://docs.fd.io/csit/rls2101/report/dpdk_performance_tests/csit_release_notes.html
[9] 
https://docs.fd.io/csit/rls2101/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] 
https://docs.fd.io/csit/rls2101/report/dpdk_performance_tests/packet_latency/index.html
[11] 
https://docs.fd.io/csit/rls2101/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18761): https://lists.fd.io/g/vpp-dev/message/18761
Mute This Topic: https://lists.fd.io/mt/80685283/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] AMD and NAT44ed stateful benchmarks in updated CSIT-2009 Report

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi All,

Quick update re recent major additions to CSIT-2009 report available at 
https://docs.fd.io/csit/rls2009/report/ :

1. Benchmark results for AMD 2n-zn2 testbed (EPYC Zen2 7532) with VPP and DPDK 
performance data:

   - Testbed description in [12]
   - Sample throughput graphs in [13]
   - Sample latency graphs in [14]
   - Sample benchmark tables in [15]
   - Sample VPP operational data (“show runtime”) in [16]

2. New NAT44 Endpoint Dependent stateful benchmarks

   - Methodology for Connections-Per-Second (CPS) and Packets-Per-Second (PPS) 
for UDP and TCP/IP [17]
   - Sample benchmark graphs UDP CPS [18], TCP CPS [19], UDP PPS [20], TCP PPS 
[21]

Happy browsing! :)

Cheers,
Maciek

[12] 
https://docs.fd.io/csit/rls2009/report/introduction/physical_testbeds.html#node-amd-epyc-zen2-2n-zn2
[13] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_throughput_graphs/ip4-2n-zn2-xxv710.html
[14] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_latency/ip4-2n-zn2-xxv710.html
[15] 
https://docs.fd.io/csit/rls2009/report/detailed_test_results/vpp_performance_results_2n_zn2/ip4_xxv710.html
[16] 
https://docs.fd.io/csit/rls2009/report/test_operational_data/vpp_performance_operational_data_2n_zn2/ip4_xxv710.html

[17] 
https://docs.fd.io/csit/rls2009/report/introduction/methodology_nat44.html#nat44-endpoint-dependent
[18] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_throughput_graphs/nat44-ed-udp-cps.html
[19] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_throughput_graphs/nat44-ed-tcp-cps.html
[20] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_throughput_graphs/nat44-ed-udp-pps.html
[21] 
https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/packet_throughput_graphs/nat44-ed-tcp-pps.html


> On 14 Oct 2020, at 21:58, Maciek Konstantynowicz (mkonstan) 
>  wrote:
> 
> Hi All,
> 
> FD.io CSIT-2009 report is available on FD.io docs site:
> 
>https://docs.fd.io/csit/rls2009/report/
> 
> Great thanks to all contributors in CSIT and VPP communities!
> 
> Below summary and pointers to specific sections in the report.
> Welcome all comments, best by email to csit-...@lists.fd.io.
> 
> Cheers,
> -Maciek
> 
> 
> CSIT-2009 Release Summary
> -
> 
> NEW TESTS
> 
> - A new category of tests using TRex ASTF stateful APIs and traffic
>  profiles with up to 16M UDP and TCP/IP sessions. Initial stateful
>  tests include VPP NAT44 Endpoint Dependent (NAT44ed)
>  connections-per-second and packets-per-second throughput (with
>  controlled packet size). (Note: report test runs are still to be
>  executed in their fullness, expect them to appear in maintenance
>  report versions next week and week after next. Maintenance reports are
>  published on a weekly basis if there are changes.)
> 
> - Refactored existing NAT44 Deterministic (NAT44det) throughput tests
>  and added higher session scale, up to 16M UDP sessions. Continue to
>  use TRex STL stateless APIs and traffic profiles.
> 
> - Added NAT44ed uni-directional UDP throughput tests using TRex STL
>  stateless APIs and traffic profiles, as a way to verify stateful tests
>  performance.
> 
> - IPsec async mode VPP performance tests, with HW crypto only for now,
>  meaning Xeon Haswell testbeds only.
> 
> - Full suite of tests now running on Mellanox ConnectX5-2p100GE NICs in
>  2n-clx (Intel Xeon Cascadelake) testbeds using VPP native rdma driver.
>  For the first time one case see linear multi-core speedup into 72 Mpps
>  region (L2 on 2 cores, IPv4 on 4 cores), in some cases NIC is the
>  limit again (like it was to date in CSIT labs with FVL 2p25GE NICs).
> 
> BENCHMARKING
> 
> - AMD 2n-zn2 testbed onboarded with EPYC 7532 32-Core Processor. Full
>  set of CSIT-2009 results to be included in one of the upcoming
>  maintenance reports, following completion of calibrating dry runs that
>  are currently ongoing.
> 
> - Optimization and calibration of TRex STL and ASTF multi-core
>  configurations, a small impact on test results as captured in current
>  vs. previous release performance comparisons.
> 
> 
> Pointers to CSIT-2009 Report sections
> -
> 
> 1. FD.io CSIT test methodology  [1]
> 2. VPP release notes[2]
> 3. VPP 64B/IMIX throughput graphs   [3]
> 4. VPP throughput speedup multi-core[4]
> 5. VPP latency under load   [5]
> 6. VPP comparisons v20.09 vs. v20.05[6]
> 7. VPP performance all pkt sizes & NICs [7]
> 8. DPDK 20.08 apps release notes[8]
> 9. DPDK 64B throughput graphs   [9]
> 10. DPDK latency under load  [10]
> 11. DPDK comparisons 20.08 vs. 20.02[11]
> 
> Functional device tests (VPP_Device) are also included in the report.
> 
> [1] https://docs.fd.io/csit/rls2009/report/introduction/methodology.html
> [2] 
> https://docs.fd.io/csit/rls2009/report/vpp_performance_tests/csit_release_notes.html
> [3] 

Re: [vpp-committers] [vpp-dev] VPP committers: VPP PTL vote

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

+csit-dev

Congrats Damjan - well done!

And Dave, huge Thank You, once again, for starting and navigating the FD.io VPP 
project with skill, strength and wit, and for making FD.io what it is today, 
with a healthily growing FD.io community and code base across a diverse group 
of members and platforms, Thank You! 

And Damjan, Good luck!!

Cheers,
Maciek

> On 28 Sep 2020, at 12:47, Damjan Marion via lists.fd.io 
>  wrote:
> 
> 
> 
> Now when we have votes from all 12 committers (me excluded) I would like to 
> thank you all for your +1s.
> It is nice to be elected by unanimous decision :)
> 
> — 
> Damjan
> 
> 
>> On 28.09.2020., at 09:44, Benoit Ganne (bganne) via lists.fd.io 
>>  wrote:
>> 
>> +1
>> 
>> And I take this opportunity to say a big thank you Dave for your efforts to 
>> build an healthy community, and all the best for Damjan in his (I hope) 
>> future new role 
>> 
>> Best
>> ben
>> 
>>> -Original Message-
>>> From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach
>>> via lists.fd.io
>>> Sent: vendredi 25 septembre 2020 21:14
>>> To: vpp-committ...@lists.fd.io
>>> Cc: vpp-dev@lists.fd.io
>>> Subject: [vpp-dev] VPP committers: VPP PTL vote
>>> 
>>> Folks,
>>> 
>>> 
>>> 
>>> The self-nomination period closed yesterday. We had one self-nomination,
>>> from Damjan Marion. At this point, we can proceed with a vote.
>>> 
>>> 
>>> 
>>> I’m sure that Damjan will do a great job, so let me start:
>>> 
>>> 
>>> 
>>> Damjan Marion as VPP PTL: +1
>>> 
>>> 
>>> 
>>> Please vote +1, 0, -1. For once, the “reply-all” button is everyone’s
>>> friend.
>>> 
>>> 
>>> 
>>> Thanks... Dave
>>> 
>>> 
>>> 
>>> -Original Message-
>>> From: d...@barachs.net 
>>> Sent: Thursday, September 17, 2020 10:32 AM
>>> To: 'vpp-dev@lists.fd.io' ; 't...@lists.fd.io'
>>> 
>>> Subject: Happy Trails to Me...
>>> 
>>> 
>>> 
>>> Folks,
>>> 
>>> 
>>> 
>>> I’m departing the employment rolls towards the end of next month. Although
>>> I intend to remain active in the fd.io vpp community as a coder,
>>> committer, and resident greybeard, it’s time for the community to pick a
>>> new PTL.
>>> 
>>> 
>>> 
>>> According to the project governance document,
>>> https://fd.io/docs/tsc/FD.IO-Technical-Community-Document-12-12-2017.pdf
>>> 
>>> :
>>> 
>>> 
>>> 
>>> 3.2.3.1 Project Technical Leader Candidates Candidates for the project’s
>>> PTL will be derived from the Committers of the Project. Candidates must
>>> self-nominate.
>>> 
>>> 
>>> 
>>> I'd like to invite any interested vpp project committer to self-nominate
>>> for the PTL role. Please email vpp-dev@lists.fd.io >> d...@lists.fd.io> .
>>> 
>>> 
>>> 
>>> Let's close the self-nomination period in one week: more specifically, by
>>> 5pm EDT on Thursday, September 24, 2020; committer vote to follow
>>> thereafter.
>>> 
>>> 
>>> 
>>> I'll be glad to answer unicast questions about the PTL role from eligible
>>> committers.
>>> 
>>> 
>>> 
>>> Thanks... Dave
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> 


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#17553): https://lists.fd.io/g/vpp-dev/message/17553
Mute This Topic: https://lists.fd.io/mt/77172336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-2005 Release Report

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Hi All,

FD.io CSIT-2005 report has been published on FD.io docs site:

https://docs.fd.io/csit/rls2005/report/

Many thanks to All in CSIT, VPP and wider FD.io community who
contributed and worked hard to make CSIT-2005 happen!

See below for pointers to specific sections in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek


Points of Note in the CSIT-2005 Report

Indexed specific links listed at the bottom.

1. VPP release notes
   a. Changes in CSIT-2005: [1]
   b. Known issues: [2]

2. VPP performance - 64B/IMIX throughput graphs (selected NIC models):
   a. Graphs explained: [3]
   b. L2 Ethernet Switching:[4]
   c. IPv4 Routing: [5]
   d. IPv6 Routing: [6]
   e. SRv6 Routing: [7]
   f. IPv4 Tunnels: [8]
   g. KVM VMs vhost-user:   [9]
   h. LXC/DRC Container Memif: [10]
   e. IPsec IPv4 Routing:  [11]
   f. Virtual Topology System: [12]

3. VPP performance - multi-core and latency graphs:
   a. Speedup Multi-Core:  [13]
   b. Latency: [14]

4. VPP performance comparisons
   a. VPP-20.05 vs. VPP-20.01:  [15]

5. VPP performance test details - all NICs:
   a. Detailed results 64B IMIX 1518B 9kB:  [16]
   b. Configuration:[17]

DPDK Testpmd and L3fwd performance sections follow similar structure.

6. DPDK applications:
  a. Release notes:   [18]
  b. DPDK performance - 64B throughput graphs:[19]
  c. DPDK performance - latency graphs:   [20]
  d. DPDK performance - DPDK-20.02 vs. DPDK-19.08: [21]

Functional device tests (VPP_Device) are also included in the report.

Specific links within the report:

 [1] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/csit_release_notes.html#changes-in-csit-release
 [2] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/csit_release_notes.html#known-issues
 [3] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/index.html
 [4] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/l2.html
 [5] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/ip4.html
 [6] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/ip6.html
 [7] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/srv6.html
 [8] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/ip4_tunnels.html
 [9] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.html
[10] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/container_memif.html
[11] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/ipsec.html
[12] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_throughput_graphs/vts.html
[13] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[14] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/packet_latency/index.html
[15] 
https://docs.fd.io/csit/rls2005/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[16] 
https://docs.fd.io/csit/rls2005/report/detailed_test_results/vpp_performance_results/index.html
[17] 
https://docs.fd.io/csit/rls2005/report/test_configuration/vpp_performance_configuration/index.html
[18] 
https://docs.fd.io/csit/rls2005/report/dpdk_performance_tests/csit_release_notes.html
[19] 
https://docs.fd.io/csit/rls2005/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[20] 
https://docs.fd.io/csit/rls2005/report/dpdk_performance_tests/packet_latency/index.html
[21] 
https://docs.fd.io/csit/rls2005/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16945): https://lists.fd.io/g/vpp-dev/message/16945
Mute This Topic: https://lists.fd.io/mt/75498644/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] LFN DCs network outage - CSIT test jobs stopped until further notice

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Dear All,

Due to ongoing network issues in LFN hosted Data Centres CSIT project is 
stopping all automated benchmarking tests until further notice.
All daily and weekly trending, as well as CSIT-2005 report test jobs are 
stopped.

LFN IT and Vexxhost teams are engaged. For latest see these two tickets:

  https://secure.vexxhost.com/billing/viewticket.php?tid=TXU-374867=GB4HFZHP
  https://jira.linuxfoundation.org/servicedesk/customer/portal/2/IT-19866

We will keep you posted on progress.

Regards,
Maciek-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16659): https://lists.fd.io/g/vpp-dev/message/16659
Mute This Topic: https://lists.fd.io/mt/74690784/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] CSIT-2001 update: Xeon Skylake Performance and Progressions/Regressions RCAs

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Slides used on today’s VPP call: 
https://wiki.fd.io/view/File:200512-csit-vpp-readout.pptx

> On 12 May 2020, at 15:18, Maciek Konstantynowicz (mkonstan) 
>  wrote:
> 
> Dear All,
> 
> We have finally pushed out an update to CSIT-2001 report with VPP
> performance data for testbeds with Intel Xeon Skylake processors (2n-skx
> and 3n-skx testbeds), with SUT and TG servers impacted by firmware and
> OS upgrades (BIOS, ucode, kernel updates with mitigations against the
> newly discovered Spectre-Meltdown security vulnerabilities).
> 
> The updated CSIT-2001 report should be available for browsing just
> before 15:00 UTC today, subject to Jenkins job execution (will have
> updated version timestamp):
> 
>https://docs.fd.io/csit/rls2001/report/
>
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/csit_release_notes.html
> 
> In addition to 2n-skx and 3n-skx performance data available at the usual
> locations in the report (see links [r1] to [r4] referenced below), we
> have expanded the way we do VPP release-to-release comparisons and root
> cause analysis (RCA) for any identified performance progressions and
> regressions:
> 
>- CSIT test environment is now versioned, with ver. 1 associated
>  with CSIT rls1908 git branch as of 2019-08-21, and ver. 2
>  associated with CSIT master and rls2001 git branches as of
>  2020-03-27.
> 
>- To identify SUT performance change(s) due to CSIT test environment
>  change(s) from ver. 1 to ver. 2, VPP v19.08.1 has been re-tested
>  in ver. 2 and results compared against the past data obtained with
>  ver. 1. RCA1 analysis has been applied to this part. See [r5].
> 
>- To identify SUT performance change(s) due to VPP code change(s)
>  from v19.08.1 to v20.01.0, both VPP versions have been tested in
>  CSIT environment ver. 2 and results compared. Separate RCA2
>  analysis has been applied to this part. See [r5].
> 
>- At this stage RCA1 and RCA2 analyses are focusing on progressions > +5%
>  and regressions < -5%.
> 
> Attached pasted complete list of RCAs identified as part of this
> exercise [1] to [12].
> 
> Hope it makes sense. For any questions and comments please contact
> csit-...@lists.fd.io.
> 
> Regards,
> Maciek
> (on behalf of FD.io CSIT team)
> 
> 
> Specific links within the report:
> 
> [r1] VPP throughput graphs,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/index.html
> 
> [r2] VPP throughput speedup multi-core,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/throughput_speedup_multi_core/ip4-2n-skx-xxv710.html
> 
> [r3] VPP packet latency,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_latency/index.html
> 
> [r4] VPP soak tests,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/soak_tests/index.html
> 
> [r5] 2n-skx PDR comparison with RCA,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/comparisons/current_vs_previous_release.html#n-skx
> 
> [r6] 3n-skx PDR comparison with RCA,
> 
> https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/comparisons/current_vs_previous_release.html#id1
> 
> RCA1:
> 
> [1] DONE, Impact of upgrades: i) Skx ucode from 0x243 to 0x265,
>[ii) Linux kernel from 4.15.0-60 to 4.15.0-72 and iii) SuperMicro
>[motherboard BIOS from 3.0c to 3.2.
> 
> [2] DONE, Applied fix of FVL NIC firmware 6.0.1 for increasing TRex pps
>rate from 27 Mpps to 37 Mpps, [CSIT-1503], [TRex-519].
> 
> [3] DONE, Applied VPP PAPI fix to enable memif zero-copy, [CSIT-1592],
>[VPP-1764].
> 
> [4] OPEN, Higher than before StDev of PDR throughput for VPP vhost-user
>with VPP-inside-VM, under investigation, [CSIT-1699], [CSIT-1704].
> 
> RCA2:
> 
> [5] OPEN, dot1q-l2xcbase progression, retro-inspection of weekly ndrpdr
>tests points to ge-22805, automated bisect script does not work
>due to frequent API changes, [CSIT-1699], [CSIT-1705].
> 
> [6] DONE, ip4base-nat44 regression, ge-23963
>(https://gerrit.fd.io/r/c/vpp/+/23963#message-044278e6_752c3327).
> 
> [7] WIP, avf-ip4scale regression, CANDIDATE(S) before ge-22699, [
>CSIT-1699], [CSIT-1706].
> 
> [8] OPEN, VPP vhost-user with VPP-inside-VM higher than before stdev
>of PDR throughput, under investigation, [CSIT-1699], [CSIT-1704].
> 
> [9] WIP, vhost-user with testpmd-in-VM progression, CANDIDATE(S)
>before 22277, [CSIT-1699], [CSIT-1707].
> 
> [10] WIP, avf-ip4base regression, CANDIDATE(S) range
> ge-18361..ge-24505, [CSIT-1699], [CSIT-1708].
> 
> [11] DONE, memif regression, CANDIDATE(S) confirmed ge-23801.
> 
> [12] WIP, ipsec tnl sw scale regression, CANDIDATE(S) before ge-23557,
> [CSIT-1699], [CSIT-1712].

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16349): https://lists.fd.io/g/vpp-dev/message/16349
Mute This Topic: https://lists.fd.io/mt/74159188/21656
Group 

[vpp-dev] CSIT-2001 update: Xeon Skylake Performance and Progressions/Regressions RCAs

[Maciek Konstantynowicz (mkonstan) via lists.fd.io]

Dear All,

We have finally pushed out an update to CSIT-2001 report with VPP
performance data for testbeds with Intel Xeon Skylake processors (2n-skx
and 3n-skx testbeds), with SUT and TG servers impacted by firmware and
OS upgrades (BIOS, ucode, kernel updates with mitigations against the
newly discovered Spectre-Meltdown security vulnerabilities).

The updated CSIT-2001 report should be available for browsing just
before 15:00 UTC today, subject to Jenkins job execution (will have
updated version timestamp):

https://docs.fd.io/csit/rls2001/report/

https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/csit_release_notes.html

In addition to 2n-skx and 3n-skx performance data available at the usual
locations in the report (see links [r1] to [r4] referenced below), we
have expanded the way we do VPP release-to-release comparisons and root
cause analysis (RCA) for any identified performance progressions and
regressions:

- CSIT test environment is now versioned, with ver. 1 associated
  with CSIT rls1908 git branch as of 2019-08-21, and ver. 2
  associated with CSIT master and rls2001 git branches as of
  2020-03-27.

- To identify SUT performance change(s) due to CSIT test environment
  change(s) from ver. 1 to ver. 2, VPP v19.08.1 has been re-tested
  in ver. 2 and results compared against the past data obtained with
  ver. 1. RCA1 analysis has been applied to this part. See [r5].

- To identify SUT performance change(s) due to VPP code change(s)
  from v19.08.1 to v20.01.0, both VPP versions have been tested in
  CSIT environment ver. 2 and results compared. Separate RCA2
  analysis has been applied to this part. See [r5].

- At this stage RCA1 and RCA2 analyses are focusing on progressions > +5%
  and regressions < -5%.

Attached pasted complete list of RCAs identified as part of this
exercise [1] to [12].

Hope it makes sense. For any questions and comments please contact
csit-...@lists.fd.io.

Regards,
Maciek
(on behalf of FD.io CSIT team)


Specific links within the report:

[r1] VPP throughput graphs,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/index.html

[r2] VPP throughput speedup multi-core,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/throughput_speedup_multi_core/ip4-2n-skx-xxv710.html

[r3] VPP packet latency,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_latency/index.html

[r4] VPP soak tests,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/soak_tests/index.html

[r5] 2n-skx PDR comparison with RCA,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/comparisons/current_vs_previous_release.html#n-skx

[r6] 3n-skx PDR comparison with RCA,
 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/comparisons/current_vs_previous_release.html#id1

RCA1:

[1] DONE, Impact of upgrades: i) Skx ucode from 0x243 to 0x265,
[ii) Linux kernel from 4.15.0-60 to 4.15.0-72 and iii) SuperMicro
[motherboard BIOS from 3.0c to 3.2.

[2] DONE, Applied fix of FVL NIC firmware 6.0.1 for increasing TRex pps
rate from 27 Mpps to 37 Mpps, [CSIT-1503], [TRex-519].

[3] DONE, Applied VPP PAPI fix to enable memif zero-copy, [CSIT-1592],
[VPP-1764].

[4] OPEN, Higher than before StDev of PDR throughput for VPP vhost-user
with VPP-inside-VM, under investigation, [CSIT-1699], [CSIT-1704].

RCA2:

[5] OPEN, dot1q-l2xcbase progression, retro-inspection of weekly ndrpdr
tests points to ge-22805, automated bisect script does not work
due to frequent API changes, [CSIT-1699], [CSIT-1705].

[6] DONE, ip4base-nat44 regression, ge-23963
(https://gerrit.fd.io/r/c/vpp/+/23963#message-044278e6_752c3327).

[7] WIP, avf-ip4scale regression, CANDIDATE(S) before ge-22699, [
CSIT-1699], [CSIT-1706].

[8] OPEN, VPP vhost-user with VPP-inside-VM higher than before stdev
of PDR throughput, under investigation, [CSIT-1699], [CSIT-1704].

[9] WIP, vhost-user with testpmd-in-VM progression, CANDIDATE(S)
before 22277, [CSIT-1699], [CSIT-1707].

[10] WIP, avf-ip4base regression, CANDIDATE(S) range
 ge-18361..ge-24505, [CSIT-1699], [CSIT-1708].

[11] DONE, memif regression, CANDIDATE(S) confirmed ge-23801.

[12] WIP, ipsec tnl sw scale regression, CANDIDATE(S) before ge-23557,
 [CSIT-1699], [CSIT-1712].-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16343): https://lists.fd.io/g/vpp-dev/message/16343
Mute This Topic: https://lists.fd.io/mt/74159188/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-2001 Release Report - CSIT-2001.09 update

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Update: CSIT-2001.09 maintenance report has been published:

https://docs.fd.io/csit/rls2001/report/

Changes from last week version include:

Added VPP performance test data into Detailed Results per testbed types:
  - 2n-clx (Xeon Cascadelake): IPv4, IPv6, L2.
  - 2n-dnv (Atom Denverton): IPsec, IPv4 tunnels.
  - 3n-hsw (Xeon Haswell): IPv4, L2, vhost.

Added VPP performance tests selected for analysis and graphs per testbed 
types:
  - 2n-clx (Xeon Cascadelake): all tests.
  - 3n-tsh (Arm Cortex-A72 Taishan): all tests.

Added DPDK performance tests selected for analysis and graphs per testbed 
types:
  - 2n-clx (Xeon Cascadelake): all tests.


2n-skx and 3n-skx testbeds (Xeon Skylake) are still out of service due to the 
processor microcode issue.
For status see:


https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/csit_release_notes.html

https://docs.fd.io/csit/rls2001/report/dpdk_performance_tests/csit_release_notes.html

Welcome all comments, best by email to 
csit-...@lists.fd.io.

Cheers,
-Maciek


On 14 Feb 2020, at 14:02, Maciek Konstantynowicz (mkonstan) 
mailto:mkons...@cisco.com>> wrote:

Hi All,

FD.io CSIT-2001 report has been published on FD.io 
docs site:

   https://docs.fd.io/csit/rls2001/report/

Many thanks to All in CSIT, VPP and wider FD.io community who
contributed and worked hard to make CSIT-2001 happen!

Below three summaries:
- Intel Xeon 2n-skx, 3n-skx and 2n-clx Testbeds microcode issue.
- CSIT-2001 Release Summary, a high-level summary.
- Points of Note in CSIT-2001 Report, with specific links to report.

Welcome all comments, best by email to 
csit-...@lists.fd.io.

Cheers,
-Maciek


NOTE: Intel Xeon 2n-skx, 3n-skx and 2n-clx Testbeds microcode issue.

   VPP and DPDK performance test data is not included in this report
   version. This is due to the lower performance and behaviour
   inconsistency of these systems following the upgrade of processor
   microcode packages (skx ucode 0x264, clx ucode 0x52c), done
   as part of updating Ubuntu 18.04 LTS kernel version. Tested VPP and
   DPDK applications (L3fwd) are affected. Skx and Clx test data will
   be added in subsequent maintenance report version(s) once the issue
   is resolved. See https://jira.fd.io/browse/CSIT-1675.


CSIT-2001 Release Summary

1. CSIT-2001 Report

  - html link: https://docs.fd.io/csit/rls2001/report/
  - pdf link: 
https://docs.fd.io/csit/rls2001/report/_static/archive/csit_rls2001.pdf

2. New Tests

  - NFV density tests with IPsec encryption between DUTs.

  - Full test coverage for VPP AVF driver for Fortville NICs.

  - VPP Hoststack TCP/IP tests with wrk, iperf3 with LDPreload tests
without and with packet loss via VPP NSIM plugin), and QUIC/UDP/IP
transport tests.

  - Mellanox ConnectX5-2p100GE NICs in 2n-clx testbeds using VPP native
rdma driver.

  - Load Balancer tests.

3. Benchmarking

  - Fully onboarded new Intel Xeon Cascadelake Testbeds with x710,
xxv710 and mcx556a-edat NIC cards.

  - Added new High Dynamic Range Histogram latency measurements.

4. Infrastructure

  - Full migration of CSIT from Python2.7 to Python3.6.


Points of Note in the CSIT-2001 Report

Indexed specific links listed at the bottom.

1. VPP release notes
  a. Changes in CSIT-2001: [1]
  b. Known issues: [2]

2. VPP performance - 64B/IMIX throughput graphs (selected NIC models):
  a. Graphs explained: [3]
  b. L2 Ethernet Switching:[4]
  c. IPv4 Routing: [5]
  d. IPv6 Routing: [6]
  e. SRv6 Routing: [7]
  f. IPv4 Tunnels: [8]
  g. KVM VMs vhost-user:   [9]
  h. LXC/DRC Container Memif: [10]
  e. IPsec IPv4 Routing:  [11]
  f. Virtual Topology System: [12]

3. VPP performance - multi-core and latency graphs:
  a. Speedup Multi-Core:  [13]
  b. Latency: [14]

4. VPP performance comparisons
  a. VPP-20.01 vs. VPP-19.08:  [15]

5. VPP performance test details - all NICs:
  a. Detailed results 64B IMIX 1518B 9kB:  [16]
  b. Configuration:[17]

DPDK Testpmd and L3fwd performance sections follow similar structure.

6. DPDK applications:
 a. Release notes:   [18]
 b. DPDK performance - 64B throughput graphs:[19]
 c. DPDK performance - latency graphs:   [20]
 d. DPDK performance - DPDK-19.08 vs. DPDK-19.05: [21]

Functional tests, 

Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi Dave, thanks for notifying, we have just discussed this on the CSIT project 
call, Vratko is looking into this.
Here are the incompatible API CRCs that are tripping the job. Will notify on 
this thread once issue is nailed down.

Cheers,
-Maciek


13:24:04 Incompatible API CRCs found in .api.json files:
13:24:04 {
13:24:04  "ipsec_interface_add_del_spd":"0x80f80cbb",
13:24:04  "ipsec_sad_entry_add_del":"0x57cc13fa",
13:24:04  "ipsec_select_backend":"0x5bcfd3b7",
13:24:04  "ipsec_spd_add_del":"0x20e89a95",
13:24:04  "ipsec_spd_entry_add_del":"0x9f384b8d",
13:24:04  "ipsec_tunnel_if_add_del":"0x2b135e68",
13:24:04  "ipsec_tunnel_if_add_del_reply":"0x5383d31f"
13:24:04 }
13:24:04 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
"ipsec_interface_add_del_spd":"0x80f80cbb",\n 
"ipsec_sad_entry_add_del":"0x57cc13fa",\n 
"ipsec_select_backend":"0x5bcfd3b7",\n "ipsec_spd_add_del":"0x20e89a95",\n 
"ipsec_spd_entry_add_del":"0x9f384b8d",\n 
"ipsec_tunnel_if_add_del":"0x2b135e68",\n 
"ipsec_tunnel_if_add_del_reply":"0x5383d31f"\n}',)


On 26 Feb 2020, at 14:44, Dave Barach (dbarach) 
mailto:dbar...@cisco.com>> wrote:

Other patches failing as well. Please investigate AYEC.

Thanks... Dave

P.S. See https://gerrit.fd.io/r/c/vpp/+/25463. No .api files involved.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15545): https://lists.fd.io/g/vpp-dev/message/15545
Mute This Topic: https://lists.fd.io/mt/71566402/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-2001 Release Report

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi All,

FD.io CSIT-2001 report has been published on FD.io docs site:

https://docs.fd.io/csit/rls2001/report/

Many thanks to All in CSIT, VPP and wider FD.io community who
contributed and worked hard to make CSIT-2001 happen!

Below three summaries:
- Intel Xeon 2n-skx, 3n-skx and 2n-clx Testbeds microcode issue.
- CSIT-2001 Release Summary, a high-level summary.
- Points of Note in CSIT-2001 Report, with specific links to report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek


NOTE: Intel Xeon 2n-skx, 3n-skx and 2n-clx Testbeds microcode issue.

VPP and DPDK performance test data is not included in this report
version. This is due to the lower performance and behaviour
inconsistency of these systems following the upgrade of processor
microcode packages (skx ucode 0x264, clx ucode 0x52c), done
as part of updating Ubuntu 18.04 LTS kernel version. Tested VPP and
DPDK applications (L3fwd) are affected. Skx and Clx test data will
be added in subsequent maintenance report version(s) once the issue
is resolved. See https://jira.fd.io/browse/CSIT-1675.


CSIT-2001 Release Summary

1. CSIT-2001 Report

   - html link: https://docs.fd.io/csit/rls2001/report/
   - pdf link: 
https://docs.fd.io/csit/rls2001/report/_static/archive/csit_rls2001.pdf

2. New Tests

   - NFV density tests with IPsec encryption between DUTs.

   - Full test coverage for VPP AVF driver for Fortville NICs.

   - VPP Hoststack TCP/IP tests with wrk, iperf3 with LDPreload tests
 without and with packet loss via VPP NSIM plugin), and QUIC/UDP/IP
 transport tests.

   - Mellanox ConnectX5-2p100GE NICs in 2n-clx testbeds using VPP native
 rdma driver.

   - Load Balancer tests.

3. Benchmarking

   - Fully onboarded new Intel Xeon Cascadelake Testbeds with x710,
 xxv710 and mcx556a-edat NIC cards.

   - Added new High Dynamic Range Histogram latency measurements.

4. Infrastructure

   - Full migration of CSIT from Python2.7 to Python3.6.


Points of Note in the CSIT-2001 Report

Indexed specific links listed at the bottom.

1. VPP release notes
   a. Changes in CSIT-2001: [1]
   b. Known issues: [2]

2. VPP performance - 64B/IMIX throughput graphs (selected NIC models):
   a. Graphs explained: [3]
   b. L2 Ethernet Switching:[4]
   c. IPv4 Routing: [5]
   d. IPv6 Routing: [6]
   e. SRv6 Routing: [7]
   f. IPv4 Tunnels: [8]
   g. KVM VMs vhost-user:   [9]
   h. LXC/DRC Container Memif: [10]
   e. IPsec IPv4 Routing:  [11]
   f. Virtual Topology System: [12]

3. VPP performance - multi-core and latency graphs:
   a. Speedup Multi-Core:  [13]
   b. Latency: [14]

4. VPP performance comparisons
   a. VPP-20.01 vs. VPP-19.08:  [15]

5. VPP performance test details - all NICs:
   a. Detailed results 64B IMIX 1518B 9kB:  [16]
   b. Configuration:[17]

DPDK Testpmd and L3fwd performance sections follow similar structure.

6. DPDK applications:
  a. Release notes:   [18]
  b. DPDK performance - 64B throughput graphs:[19]
  c. DPDK performance - latency graphs:   [20]
  d. DPDK performance - DPDK-19.08 vs. DPDK-19.05: [21]

Functional tests, including VPP_Device (functional device tests),
VPP_VIRL and HoneyComb are all included in the report.

Specific links within the report:

 [1] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/csit_release_notes.html#changes-in-csit-release
 [2] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/csit_release_notes.html#known-issues
 [3] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/index.html
 [4] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/l2.html
 [5] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/ip4.html
 [6] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/ip6.html
 [7] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/srv6.html
 [8] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/ip4_tunnels.html
 [9] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/vm_vhost.html
[10] 
https://docs.fd.io/csit/rls2001/report/vpp_performance_tests/packet_throughput_graphs/container_memif.html
[11] 

Re: [vpp-dev] [csit-dev] FDIO Maintenance - 2020-02-05 1900 UTC to 2400 UTC

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi Vanessa,

Many thanks for the heads-up.

Unfortunately 05-Feb hits is in the middle of CSIT-2001 release, per published 
schedule here: https://wiki.fd.io/view/CSIT/csit2001_plan

Is there any chance to postpone this upgrade till after 19-Feb, which is +1 
week after our report publish target in case we slip due to infra issues (we 
got affected by more SSD failures as you may know).

Pls let us know if this is possible.

Regards,
-Maciek

On 7 Jan 2020, at 14:30, Vanessa Valderrama 
mailto:vvalderr...@linuxfoundation.org>> wrote:





Please let us know as soon as possible if this maintenance conflicts with your 
project.

What:

  *   Jenkins
 *   OS and security updates
 *   Upgrade to 2.204.1
 *   Plugin updates
  *   Nexus
 *   OS updates
  *   Jira
 *   OS updates
  *   Gerrit
 *   OS updates
  *   Sonar
 *   OS updates
  *   OpenGrok
 *   OS updates

When:  2020-02-05 1900 UTC to 2400 UTC

Impact:

Maintenance will require a reboot of each FD.io system. Jenkins 
will be placed in shutdown mode at 1800 UTC. Please let us know if specific 
jobs cannot be aborted.
The following systems will be unavailable during the maintenance window:

  *   Jenkins sandbox
  *   Jenkins production
  *   Nexus
  *   Jira
  *   Gerrit
  *   Sonar
  *   OpenGrok

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#3866): https://lists.fd.io/g/csit-dev/message/3866
Mute This Topic: https://lists.fd.io/mt/69502829/675185
Group Owner: csit-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/csit-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15097): https://lists.fd.io/g/vpp-dev/message/15097
Mute This Topic: https://lists.fd.io/mt/69532590/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [tsc] FD.io Production Jenkins Restart Required

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi Vanessa, Thanks for the note. CSIT project keeps experiencing issues
due to Jenkins outages. Do you have ETA for the fix that will stop these
outages?

-Maciek

> On 5 Nov 2019, at 23:18, Vanessa Valderrama  
> wrote:
> 
> Jenkins has been restarted, job views restored, jobs are running.
> 
> We will continue to investigate the Gateway Timeout and JNLP errors
> we've been seeing the last couple of days.
> 
> If you experience any issues, please open a ticket at
> support.linuxfoundation.org
> 
> Thank you,
> Vanessa
> 
> 
> On 11/5/19 4:39 PM, Vanessa Valderrama wrote:
>> We continue having issues with Gateway Timeouts on the CI merge job which has
>> corrupted the Jenkins job views.
>> 
>> Jenkins will need to be restarted to resolve this issue.
>> 
>> Thank you,
>> Vanessa
>> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#1152): https://lists.fd.io/g/tsc/message/1152
> Mute This Topic: https://lists.fd.io/mt/42686762/675185
> Group Owner: tsc+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/tsc/unsub  [mkons...@cisco.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14529): https://lists.fd.io/g/vpp-dev/message/14529
Mute This Topic: https://lists.fd.io/mt/44257110/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-1908 Release Report

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi All,

FD.io CSIT-1908 report has been published on FD.io docs site:

html: https://docs.fd.io/csit/rls1908/report/
pdf: https://docs.fd.io/csit/rls1908/report/_static/archive/csit_rls1908.pdf

Many thanks to All in CSIT, VPP and wider FD.io community who
contributed and worked hard to make CSIT-1908 happen!

Below two summaries:
- CSIT-1908 Release Summary, a high-level summary.
- Points of Note in CSIT-1908 Report, with specific links to report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek


CSIT-1908 Release Summary

1. CSIT-1908 Report

- html link: https://docs.fd.io/csit/rls1908/report/
- pdf link: 
https://docs.fd.io/csit/rls1908/report/_static/archive/csit_rls1908.pdf

2. New Tests

  - VM/VNF service chains with DPDK Testpmd and VPP L2/IPv4 workloads
and external VXLAN encapsulation.
  - IPsec with new VPP native cipher algorithms, baseline and large
scale (up to 60k tunnels).
  - VM/VNF service chains with VPP IPsec workloads, baseline and
horizontal scaling (experimental, in-testing).
  - GBP (Group Based Policy) with external dot1q encapsulation.
  - Extended test coverage with VPP native AVF driver: IPv4 scale tests.
  - Refreshed VPP TCP/HTTP tests.
  - Number of VPP functional device tests running in container based
environment.
  - Good VPP PAPI (Python API) test coverage, PAPI used for all VPP
tests.

3. Benchmarking

  - Added new processor micro-architectures: ARM/AArch64 (TaiShan) and
Atom (Denverton).

- New testbeds onboarded into FD.io CSIT CI/CD functional and
  performance test pipelines.
- Daily trending with throughput changes monitoring, analytics and
  anomaly auto-detection.
- Release reports with benchmarking data including throughput,
  latency, test repeatibility.

  - Updated CSIT benchmarking report specification

- Consistent selection of tests across all testbeds and processor
  microarchitectures present in FD.io labs (Xeon, Atom, ARM) for
  iterative benchmarking tests to verify results repeatibility. Data
  presented in graphs conveying NDR (non-drop rate, zero packet
  loss) and PDR (partial drop rate) throughput statistics.
  Multi-core speedup and latency are also presented.
- Consistent comparison of NDR and PDR throughput results across the
  releases.
- Updated graph naming and test grouping to improve browsability and
  access to test data.

  - Increased test coverage in 2-node testbed environment (2n-skx).

  - Updated soak testing methodology and new results, aligned with
latest IETF draft specification draft-vpolak-bmwg-plrsearch-02.

4. Infrastructure

- API
  - PAPI (Python API) used for all VPP tests, migrated away from VAT
(VPP API Test).
  - VPP API change detection and gating in VPP and CSIT CI/CD.

- Test Environments
  - VPP functional device tests: migrated away from VIRL (VM based) to
container environment (with Nomad).
  - Added new physical testbeds: ARM/AArch64 (TaiShan) and Atom
(Denverton).

- CSIT Framework
  - Conifguration keyword alignment across 2-node and 3-node testbeds to
ease test portability across environments.

- Installer
  - Updated bare-metal CSIT performance testbed installer (ansible).


Points of Note in CSIT-1908 Report

Indexed specific links listed at the bottom.

1. VPP release notes
   a. Changes in CSIT-1908: [1]
   b. Known issues: [2]

2. VPP performance - 64B/IMIX throughput graphs (selected NIC models):
   a. Graphs explained: [3]
   b. L2 Ethernet Switching:[4]
   c. IPv4 Routing: [5]
   d. IPv6 Routing: [6]
   e. SRv6 Routing: [7]
   f. IPv4 Tunnels: [8]
   g. KVM VMs vhost-user:   [9]
   h. LXC/DRC Container Memif: [10]
   e. IPsec IPv4 Routing:  [11]
   f. Virtual Topology System: [12]

3. VPP performance - multi-core and latency graphs:
   a. Speedup Multi-Core:  [13]
   b. Latency: [14]

4. VPP system performance - NFV service density and TCP/IP:
   a. VNF (VM) Service Chains:  [15]
   b. CNF (Container) Service Chains:   [16]
   c. CNF (Container) Service Pipelines:[17]
   d. HTTP and TCP/IP:  [18]

4. VPP performance comparisons
   a. VPP-19.08 vs. VPP-19.04:  [19]

5. VPP performance test details - all NICs:
   a. Detailed results 64B IMIX 1518B 9kB:  [20]
   b. Configuration:[21]

DPDK Testpmd and L3fwd performance sections follow similar structure.

6. DPDK applications:
  a. Release notes:   [22]
  b. DPDK performance - 64B throughput graphs:[23]
  c. DPDK performance - 

[vpp-dev] CSIT-1908 draft report preview available, need help with two VPP performance regressions

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi,

A draft version of CSIT-1908 test and benchmarking report is available
on the CSIT master report link:

  https://docs.fd.io/csit/master/report/index.html

Most of the selected benchmarking tests have been completed including:

- All Xeon Skylake and Haswell NDR and PDR tests - all in draft report.
- Most of Atom/Denverton NDR and PDR tests - to be added to draft report.
- Most of Arm/Taishan NDR and PDR tests - to be added to draft report.

Comparison tables show some Skylake NDR/PDR throughput regressions and
progressions vs. CSIT-1904:

- 
https://docs.fd.io/csit/master/report/vpp_performance_tests/comparisons/current_vs_previous_release.html

Their latest analysis is tracked in the two CSIT gerrit commits:

- csit1908_perf_analysis_2n_skx.md: https://gerrit.fd.io/r/c/csit/+/21637/
- csit1908_perf_analysis_3n_skx.md: https://gerrit.fd.io/r/c/csit/+/21641/

Most have been root caused to CSIT and are being addressed in CSIT
before CSIT-1908 report is published on Wednesday 04-Sep.

Two VPP packet paths, l2 bridge-domain and iacl statefull have
suspected cause: *vpp binary code change affecting throughput*.
Any assistance with these two is appreciated.

Cheers,
-Maciek-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13881): https://lists.fd.io/g/vpp-dev/message/13881
Mute This Topic: https://lists.fd.io/mt/33111523/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] FD.io Jenkins Maintenance: 2019-08-29 @ 1700 UTC and 2019-08-30 @ 1700 UTC

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Thank you Vanessa for this good news!
We are aiming to complete the critical part of the CSIT-1908 testing
jobs by 04-Sep, will keep you and the team posted.

Cheers,
-Maciek

On 29 Aug 2019, at 18:18, Vanessa Valderrama 
mailto:vvalderr...@linuxfoundation.org>> wrote:


Sandbox upgrade is complete

On 08/29/2019 10:10 AM, Vanessa Valderrama wrote:

Maintenance update

Jenkins production - The upgrade is not considered critical. We'll be 
rescheduling to allow CSIT to finish the 19.08 release.

Jenkins sandbox - Today at 1700 UTC

Thank you,
Vanessa


On 08/26/2019 11:36 AM, Vanessa Valderrama wrote:

Maintenance Reminder

On 08/22/2019 12:00 PM, Vanessa Valderrama wrote:


What:

  *   Jenkins upgrade to version 2.176.3 LTS
  *   Jenkins OS updates
  *   Jenkins plug-in updates

When:
Jenkins sandbox - 2019-08-29 @ 1700 UTC
Jenkins production - 2019-08-30 @ 1700 UTC

Impact:
This maintenance requires a reboot of Jenkins.  Jenkins will be placed in 
shutdown mode at 1600 UTC. We will terminate jobs at 1700 UTC


  *   Jenkins sandbox
 *   Status page - https://status.linuxfoundation.org/incidents/96lw8jk1xpxp
  *   Jenkins production
 *   Status page - https://status.linuxfoundation.org/incidents/4hzxy36qm7y1

Why:
Jenkins version 2.176.3 LTS will contain fixes for high severity security 
issues present in the current version of Jenkins.

  *   Jenkins Security Advisories - https://jenkins.io/security/advisories/
  *   Jenkins Changelog - https://jenkins.io/changelog-stable/




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#3700): https://lists.fd.io/g/csit-dev/message/3700
Mute This Topic: https://lists.fd.io/mt/32991954/675185
Group Owner: csit-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/csit-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13877): https://lists.fd.io/g/vpp-dev/message/13877
Mute This Topic: https://lists.fd.io/mt/33071299/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP 19.08 release is available!

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Congrats to All involved in vpp-dev and FD.io on timely and 
feature-rich VPP v19.08 release!

Andrew, congrats on 19.08 and best of luck for the next 10 yrs! ;)

Cheers,
-Maciek

On 21 Aug 2019, at 22:19, Damjan Marion via Lists.Fd.Io 
mailto:dmarion=me@lists.fd.io>> wrote:



On 21 Aug 2019, at 22:57, Andrew Yourtchenko 
mailto:ayour...@gmail.com>> wrote:



So you are in release management business next 10 years, good :)

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13805): https://lists.fd.io/g/vpp-dev/message/13805
Mute This Topic: https://lists.fd.io/mt/32983052/675185
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13815): https://lists.fd.io/g/vpp-dev/message/13815
Mute This Topic: https://lists.fd.io/mt/32983052/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-1904 Benchmarking Report

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi All,

FD.io CSIT-1904 report has been published on FD.io docs site:

html: https://docs.fd.io/csit/rls1904/report/
pdf: https://docs.fd.io/csit/rls1904/report/_static/archive/csit_rls1904.pdf

Many thanks to All in CSIT, VPP and FD.io community who contributed and
worked hard to make CSIT-1904 happen!

FD.io VPP points of note in the report:

1. VPP release notes
   a. Changes in CSIT-1904: http://bit.ly/309V2oJ
   b. Known issues: http://bit.ly/2Q2K5AM

2. VPP performance graphs
   a. Throughput 64B frames: http://bit.ly/2VrdASG
   b. Speedup Multi-Core: http://bit.ly/2LAhjbS
   c. Latency: http://bit.ly/307AQUl

3. VPP performance comparisons
   a. VPP-19.04 vs. VPP-19.01: http://bit.ly/30ci12w
   b. Xeon Skylake vs. Xeon Haswell: http://bit.ly/2PWhLzT

4. VPP performance test details
   a. Detailed results 64B IMIX 1518B 9kB: http://bit.ly/2JuTjV3
   b. Configuration: http://bit.ly/2E4tvvA

DPDK Testpmd and L3fwd performance sections follow similar structure.

1. DPDK release notes:
  a. Changes in CSIT-1904: http://bit.ly/2VrOSBK
  b. Known issues: http://bit.ly/2VuO3ry

2. DPDK performance graphs for Testpmd and L3fwd demonstrator apps
  a. Throughput 64B frames: http://bit.ly/2Lxwa72
  b. Latency: http://bit.ly/2DZoKmO

Functional tests, including VPP_Device (functional device tests),
VPP_VIRL and HoneyComb are all included in the report.

Welcome all Comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek
On behalf of FD.io CSIT Project
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12989): https://lists.fd.io/g/vpp-dev/message/12989
Mute This Topic: https://lists.fd.io/mt/31579564/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP 19.04 Release is complete!

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Great work All!

On 24 Apr 2019, at 23:06, Dave Wallace 
mailto:dwallac...@gmail.com>> wrote:

Folks,

I am pleased to announce that the VPP 19.04 Release is complete and the release 
artifacts are available on PackageCloud: https://packagecloud.io/fdio/release

Many thanks to all of the VPP & CSIT contributors and extended 
FD.io community members who helped make this another successful 
VPP release.

Cheers,
-daw-
"Your Friendly VPP 19.04 Release Manager"
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12849): https://lists.fd.io/g/vpp-dev/message/12849
Mute This Topic: https://lists.fd.io/mt/31336639/675185
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12852): https://lists.fd.io/g/vpp-dev/message/12852
Mute This Topic: https://lists.fd.io/mt/31336639/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: Tech paper - "Benchmarking SW Data Planes Intel Xeon Skylake vs. Broadwell"

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi,

We finally managed to get the go ahead to publish above technical paper.

  Workloads tested: VPP, OVS-DPDK, DPDK(testpmd,l3fwd), CoreMark.
  Processors tested: Intel Xeon Skylake vs. Broadwell.
  Compared side-by-side: performance and efficiency of workloads/processors.

Thanks to all involved in peer reviews!

Here are the LF links:

  LFN sites:
On "Resources" page: 
  https://www.lfnetworking.org/resources/ 
Direct pdf download link: 
  
https://www.lfnetworking.org/wp-content/uploads/sites/55/2019/03/benchmarking_sw_data_planes_skx_bdx_mar07_2019.pdf
 

  FD.io sites:
On "Resources" page: 
  https://fd.io/resources/
Direct pdf download link: 
  
https://fd.io/wp-content/uploads/sites/34/2019/03/benchmarking_sw_data_planes_skx_bdx_mar07_2019.pdf

It is a sequel to our original “Benchmarking and Analysis of Software Data 
Planes” paper published back in December 2017:
  
https://fd.io/wp-content/uploads/sites/34/2018/01/performance_analysis_sw_data_planes_dec21_2017.pdf

Welcome all comments, send them to authors :)

Cheers,
-Maciek-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12498): https://lists.fd.io/g/vpp-dev/message/12498
Mute This Topic: https://lists.fd.io/mt/30403099/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [csit-dev] [vpp-dev] Heads up: API cleanup

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

// re-adding vpp-dev, Neale, Ole

Vratko, thanks for spotting!
Spoke with Neale, he confirmed route add/del API changes should be transparent 
to VAT per his updates in [1].
If you spotted any VatExecutor breakage, then this would need to be fixed 
before [2] is merged.
And as CSIT embarked on VAT to PAPI migration, it makes sense to use the new 
PAPI syntax for all PAPI L1 keywords listed in [3].
Neale is kind to give here heads-up before merging [2], most likely next week.
Cheers,
-Maciek

[1] https://gerrit.fd.io/r/#/c/12296/29/src/vat/api_format.c
[2] https://gerrit.fd.io/r/#/c/12296/
[3] https://git.fd.io/csit/tree/resources/libraries/python

On 28 Feb 2019, at 12:19, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco) via Lists.Fd.Io 
mailto:vrpolak=cisco@lists.fd.io>> wrote:

I see this having a big impact on our tests
(breaking VatExecutor, needing to update L1 keywords with PapiExecutor).
The change is big, look at .api files.
For example, I see ip_add_del_route
is being renamed to ip_route_add_del.

I believe we should start preparing in advance.

Vratko.

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Neale Ranns via 
Lists.Fd.Io
Sent: Thursday, 2019-February-28 09:52
To: Ole Troan mailto:otr...@employees.org>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Heads up: API cleanup


Hi,

In the spirit of this work I’d like to propose a change to the route add/del 
APIs to make use of the fib_path structure. The fib_path structure, which 
describes how to deal which the matched packets, is consistent across each of 
the route types; IP, MPLS, BIER and ABF.
By using a fib_path we can also pass more than one path for each route update, 
which is how most IP unicast protocols would normally operate. It is notably 
faster to add the route once with multiple paths, than add the route multiple 
times with one path.

  https://gerrit.fd.io/r/#/c/12296/

regards,
neale


De : mailto:vpp-dev@lists.fd.io>> au nom de Ole Troan 
mailto:otr...@employees.org>>
Date : lundi 25 février 2019 à 10:02
À : vpp-dev mailto:vpp-dev@lists.fd.io>>
Objet : [vpp-dev] Heads up: API cleanup

Apologies in advance. This is going to be painful for everyone using the API. 
In the neverending saga of cleaning up the APIs and making types more explicit, 
here are the changes I’d like to get in for 19.04:

u32 sw_if_index -> vl_api_interface_index_t

u8 ip4_address[4] -> vl_api_ip4_address_t

u8 ip6_address[16] -> vl_api_ip6_address_t

u8 is_ip6
u8 address[16] -> vl_api_address_t

u8 prefix_len
u8 prefix[4/16] -> vl_api_ip4_prefix_t / vl_api_ip6_prefix_t / vl_api_prefix_t

u8 is_ -> bool

u8 name[64] -> string name

u8 mac_address[6] -> vl_api_mac_address_t

u8 data[0]
u32 length -> u8 data[length]

The explicit types allow for much better type checking on the client side, as 
well as automatic mapping into the respective types on the client side. E.g. 
vl_api_address_t mapping into Python IPAddress object.

Best regards,
Ole
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12335): https://lists.fd.io/g/vpp-dev/message/12335
Mute This Topic: https://lists.fd.io/mt/30035998/675192
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[nra...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12385): https://lists.fd.io/g/vpp-dev/message/12385
Mute This Topic: https://lists.fd.io/mt/30035998/870305
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[vrpo...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#3352): https://lists.fd.io/g/csit-dev/message/3352
Mute This Topic: https://lists.fd.io/mt/30162681/675185
Group Owner: csit-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/csit-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12404): https://lists.fd.io/g/vpp-dev/message/12404
Mute This Topic: https://lists.fd.io/mt/30173719/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Published: FD.io CSIT-1901 Open-Source Benchmarking Report

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi All,

FD.io CSIT-1901 report has been published on FD.io docs site:

 html: https://docs.fd.io/csit/rls1901/report/
 pdf: https://docs.fd.io/csit/rls1901/report/_static/archive/csit_rls1810.pdf

Many thanks to All in CSIT, VPP and FD.io community who contributed, worked
very hard to make CSIT-1901 happen!

FD.io VPP points of note in the report:

 1. VPP release notes
a. Changes in CSIT-1901: http://bit.ly/2tnfF1p
b. Known issues: http://bit.ly/2I9rWAO

 2. VPP performance graphs
a. Throughput 64B frames: http://bit.ly/2EbYxC5
b. Speedup Multi-Core: http://bit.ly/2Bw45FR
c. Latency: http://bit.ly/2SxeDPm

 3. VPP performance comparisons
a. VPP-19.01 vs. VPP-18.10: http://bit.ly/2EbrTAs
b. 3-Node Skylake vs. 3-Node Haswell testbeds: http://bit.ly/2tng7g7

 4. VPP performance test details
a. Detailed results 64B IMIX 1518B 9kB: http://bit.ly/2V2pEVV
b. Configuration: http://bit.ly/2EbyACE
c. Run-time telemetry: http://bit.ly/2tkqG3L

DPDK Testpmd and L3fwd performance sections follow similar structure.

 1. DPDK release notes:
   a. Changes in CSIT-1901: http://bit.ly/2SDRvyL
   b. Known issues: http://bit.ly/2DCCT8A

 2. DPDK performance graphs for Testpmd and L3fwd demonstrator apps
   a. Throughput 64B frames: http://bit.ly/2BBzrLj
   b. Latency: http://bit.ly/2IfiUCu

Functional tests, including VPP_Device (functional device tests),
VPP_VIRL, HoneyComb and DMM are all included in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek
On behalf of FD.io CSIT Open-Source Benchmarking Project.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#12264): https://lists.fd.io/g/vpp-dev/message/12264
Mute This Topic: https://lists.fd.io/mt/29856691/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io CSIT-18.10.w47 Weekly Maintenance Report Update

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

FD.io CSIT-18.10.w47 weekly maintenance report has been published on
FD.io docs site:

 html: https://docs.fd.io/csit/rls1810/report/
 pdf: https://docs.fd.io/csit/rls1810/report/_static/archive/csit_rls1810.pdf

Changes from last week .w46 version:

1. Added automated wrapping of long test names in legend to graphs.
2. Updated data and time format in the header.
3. Updated report versioning.
4. Added more test runs:
   a. HoneyComb Functional.
   b. VPP on 3n-hsw testbed.
5. Added more Current vs. Previous release performance comparisons:
   a. VPP 3n-skx: http://bit.ly/2R18ypw
   b. VPP 2n-skx: http://bit.ly/2A6RCqo
   c. DPDK 3n-skx: http://bit.ly/2Q6QVrp
6. Added performance comparisons between testbed types:
   a. VPP: 2n-skx vs. 3n-skx: http://bit.ly/2FzC84q
   b. DPDK: 2n-skx vs. 3n-skx: http://bit.ly/2OX9rxx
7. Added results for 2n-dnv Atom Denverton tests:
   a. VPP Packet Throughput: http://bit.ly/2Q8wdaA
   b. VPP MRR: http://bit.ly/2FwSj2b

All per section links listed below stay unchanged.

FD.io VPP points of note in the report:

 1. VPP release notes
a. Changes in CSIT-18.10: http://bit.ly/2OxSUA6
b. Known issues: http://bit.ly/2REvB9D

 2. VPP performance graphs
a. Throughput: http://bit.ly/2OwQPEn
b. Speedup Multi-Core: http://bit.ly/2zztqgi
c. Latency: http://bit.ly/2F9j4K6

 3. VPP performance comparisons
a. VPP-18.10 vs. VPP-18.07: http://bit.ly/2RGBvXM
b. 3-Node Skylake vs. 3-Node Haswell testbeds: http://bit.ly/2DqjrOJ
c. 2-Node Skylake vs. 3-Node Skylake testbedds: http://bit.ly/2FzC84q

 4. VPP performance test details
a. Detailed results: http://bit.ly/2yVWMGf
b. Configuration: http://bit.ly/2qy7w8U
c. Run-time telemetry: http://bit.ly/2PQlkKw

DPDK Testpmd and L3fwd performance sections follow similar structure.

 1. DPDK release notes:
   a. Changes in CSIT-18.10: http://bit.ly/2Oy7E1U
   b. Known issues: http://bit.ly/2RGGE22

 2. DPDK performance graphs for Testpmd and L3fwd demonstrator apps
   a. Throughput: http://bit.ly/2PeSlAz
   b. Latency: http://bit.ly/2z5OSdx

 3. DPDK performance comparisons
a. DPDK-18.08 vs. DPDK-18.05: http://bit.ly/2Q6QVrp

Functional tests, including VPP_Device (functional device tests),
VPP_VIRL, HoneyComb, NSH_SFC and DMM are all included in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11351): https://lists.fd.io/g/vpp-dev/message/11351
Mute This Topic: https://lists.fd.io/mt/28277551/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io compute infra down -

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Nexus(?) backend to nginx is reporting down.
Resolution time?

Cheers,
-Maciek

[cid:876751CF-C733-46B9-9A07-8298C0A0A0C8@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11286): https://lists.fd.io/g/vpp-dev/message/11286
Mute This Topic: https://lists.fd.io/mt/28166437/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io CSIT Project weekly meeting - UTC GMT time change, UK and EU no change

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi,

Just wanted to make everyone aware that today’s call is taking 
place at 15:00-16:00 UTC, as UK and Europe moved away from Daylight
Saving Time (BST, CEST respectively), and looking thru the window
Summer time is definitely over, so time to change our regular habits :)

Updated details are in the usual place, on FD.io wiki:

https://wiki.fd.io/view/CSIT/Meeting

It is LFN FD.io wiki that is used to coordinate FD.io sub-projects'
events, but as it’s not automated nor intelligent, it doesn’t know
human time changed in some earthly TZs, but didn’t in other ones.

And as a critical mass of committers lives in England and Europe
geo locations, it’s is logical and functional, that meeting time
follows those geo locations. 

Cheers,
-Maciek

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#11053): https://lists.fd.io/g/vpp-dev/message/11053
Mute This Topic: https://lists.fd.io/mt/27805352/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VAT api changes 5507192..8807674

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Is it still broken?

-Maciek

On 3 Oct 2018, at 09:11, Peter Mikus via Lists.Fd.Io 
mailto:pmikus=cisco@lists.fd.io>> wrote:

Hello devs,

Recently in CSIT we are facing the issue with vpp_api_test console with JSON 
enabled, when calling dump_interface_table:

dump_interface_table:6017: JSON output supported only for VPE API calls and 
dump_stats_table
main:427: BUG: message reply spin-wait timeout

our API call:

$ sudo -S vpp_api_test json in dump_interfaces.vat script

Where:
$ cat dump_interfaces.vat
sw_interface_dump
dump_interface_table
quit

Brief bisecting reveals that its happening after one of these:

$ git log --pretty=oneline 5507192..8807674
88076749e663e35925c2212eb79e2ec4ce023772 (HEAD -> master, origin/master, 
origin/HEAD) Enabled untagged vs default functionality Removed 0-tags attribute 
for default-sub-if config Moved default-sub-if check before untagged
819d5fdb39526386ee8fe4a8729f960e84443cbd VPP-1440: clean up coverity warnings
bf49590c07162be44b21d0e0440e7fb96b2746d5 Stats: vpp_prometheus_export fixes.
94495f2a6a68ac2202b7715ce09620f1ba6fe673 PAPI: Use UNIX domain sockets instead 
of shared memory
84db4087fa38b8d4c62cbb0787d600950638034c vcl: fix coverity warning
c67cfd22bedf5b45e82ff6820d1d4e6f61ba5187 ip4-local: classify protos that skip 
csum and src check
2f54c27f7fd0d2c24e7d6b1d48809e8b58ec1abf vhost-user: add support for vlib_log 
API

Could you please take a look and let us know if this is bug or desired?

Thank you.

Peter Mikus
Engineer – Software
Cisco Systems Limited

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10737): https://lists.fd.io/g/vpp-dev/message/10737
Mute This Topic: https://lists.fd.io/mt/26716121/675185
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10752): https://lists.fd.io/g/vpp-dev/message/10752
Mute This Topic: https://lists.fd.io/mt/26716121/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io talks at ONS EU in AMS last week

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

FYI, here are the Compressed PDF copies of slides presented at EU Open Network 
Summit in Amsterday.
Some material has been out there but got adopted. Some is nett new, produced by 
speakers/authors.
On-site response was good, let’s see what the survey says.. if we ever see the 
results.
If we’re not invited back we will know though ;)

Posted on FD.io presentations side:

1. Accelerate the Development of Cloud-native VNFs - Giles Heron & Maciek 
Konstantynowicz, Cisco
  Material by: Giles, Maciek, Damjan Marion et al.
  Wednesday, September 26 • 13:50 - 14:20
  https://onseu18.sched.com/event/GwE9
  https://wiki.fd.io/view/File:Ons_eu_cloud_native_vnfs_26sep2018-pdf-min.pdf

2. High Performance Cloud Native Networking: K8s Unleashing FD.io - Maciek 
Konstantynowicz & Giles Heron, Cisco 
  Material by: Giles, Maciek, Jerome Tollet et al.
  Thursday, September 27 • 13:50 - 14:20
  https://onseu18.sched.com/event/Fms5
  
https://wiki.fd.io/view/File:Ons_eu_k8s_unleashing_fdio_27sep2018-fdio-pdf-min.pdf

Both linked from main FD.io presentations wiki:
  https://wiki.fd.io/view/Presentations (scroll to bottom of the table)

Cheers,
-Maciek

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10721): https://lists.fd.io/g/vpp-dev/message/10721
Mute This Topic: https://lists.fd.io/mt/26441707/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FD.io CSIT-18.07 v1.0 report published

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi All,

FD.io CSIT-18.07 report has been published on FD.io docs site:

  html: https://docs.fd.io/csit/rls1807/report/
  pdf: https://docs.fd.io/csit/rls1807/report/_static/archive/csit_rls1807.pdf

Many thanks to All in CSIT and FD.io community who contributed and made
CSIT-18.07 happen!

CSIT-18.07 is the first release able to use the brand new FD.io labs and
testing servers equipped the latest Intel Xeon Skylake processors. See
point 3. below for throughput comparison between Skylake and Haswell
testbeds.
This is finally making VPP code shine in a modern compute world :)

The labs were long time in the making and they have finally
materialized. Thanks again to All who made it happen!

CSIT-18.07 report includes much more test results data and report html
layout got revamped quite a bit, striving to improve presentation
clarity and browsability. Test environment descriptions also got a
substantial makeover.

VPP points of note in the report:

  1. VPP release notes
a. Changes in CSIT-18.07 [1]
b. Known issues [2]

  2. VPP performance graphs
a. Throughput [3]
b. Speedup Multi-Core [4]
c. Latency [5]

  3. VPP performance comparisons
b. VPP-18.07 vs. VPP-18.04 [6]
c. 3-Node Skylake vs. 3-Node Haswell testbed types [7]

  4. VPP performance test details
a. Results [8]
b. Configuration [9]
c. Run-time telemetry [10]

DPDK Testpmd and L3fwd performance sections follow similar structure.

Functional tests, including VPP, HoneyComb, NSH_SFC and DMM are all
included in the report.

Welcome all comments, best by email to csit-...@lists.fd.io.

Cheers,
-Maciek
On behalf of FD.io CSIT.

[1] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/csit_release_notes.html#changes-in-csit-release
[2] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/csit_release_notes.html#known-issues

[3] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/packet_latency_graphs/index.html

[6] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] 
https://docs.fd.io/csit/rls1807/report/vpp_performance_tests/comparisons/3n-skx_vs_3n-hsw_testbeds.html

[8] 
https://docs.fd.io/csit/rls1807/report/detailed_test_results/vpp_performance_results/index.html
[9] 
https://docs.fd.io/csit/rls1807/report/test_configuration/vpp_performance_configuration/index.html
[10] 
https://docs.fd.io/csit/rls1807/report/test_operational_data/vpp_performance_operational_data/index.html-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#10212): https://lists.fd.io/g/vpp-dev/message/10212
Mute This Topic: https://lists.fd.io/mt/24677295/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [csit-dev] [vpp-dev] Parallel test execution in VPP Test Framework

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

+1

-Maciek

On 28 Jul 2018, at 13:28, Damjan Marion mailto:dmar...@me.com>> 
wrote:


Dear All,

My personal preference is that make test  framework implements cpu assignment 
code.
It should't be rocket science to parse /sys/devices/system/cpu/online and give 
one cpu to each instance.
It will also help to test framework to understand how many parallel jobs it can 
run...

Enforcing single cpu assignment in vpp is done intentionally, to avoid cross 
numa memory allocation.
If main-core is not specified, vpp simply uses cpu core 1 (unless only 0 
exists).
While adding something like "cpu { main-core any} " should be quite straight 
forward, it will have broken
behaviour when dpdk is loaded and it will just confuse people. Also, we will 
need to come back to the
drawing board when we decide to run multiple workers in make test, as logic 
there is more complex and will likely require
rework of the thread placement code.

--
Damjan

On 27 Jul 2018, at 20:46, Peter Mikus via Lists.Fd.Io 
mailto:pmikus=cisco@lists.fd.io>> wrote:

Hello,

>  What is the “significant problem” you’re running into?

The problem can be better described as: When python is spawning N instances of 
VPP process, all processes are from unknown reason placed with affinity 0x2 
(bin 10). This can be verified by taskset –p . CFS is then placing all 
VPP process to the same core, making it inefficient on multicore jenkins slave 
container.
The default vpp startup.conf is not modified thus there is no input to know 
where to pin the vpp threads. Simply one can said or think that this is related 
to python multiprocess/subprocess.popen code, which is hard-setting affinity 
mask to 0x2.

There are multiple solutions for workaround that Juraj proposed or Maciek, but 
none of them is answering why is this happening.

Peter Mikus
Engineer – Software
Cisco Systems Limited

From: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io> 
[mailto:csit-...@lists.fd.io] On Behalf Of Maciek Konstantynowicz (mkonstan) 
via Lists.Fd.Io
Sent: Friday, July 27, 2018 6:53 PM
To: Alec Hothan (ahothan) mailto:ahot...@cisco.com>>; Juraj 
Linkeš mailto:juraj.lin...@pantheon.tech>>
Cc: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>
Subject: Re: [csit-dev] [vpp-dev] Parallel test execution in VPP Test Framework

Alec, This is about make test and not real packet forwarding. Per Juraj’s patch 
[1]

Juraj, My understanding is that if you’re starting VPP without specifying core 
placement in startup.conf [2] cpu {..}, then Linux CFS will be placing the 
threads onto available cpu core resources. If you’re saying this is not the 
case, and indeed the wiki comment indicates this, then the way to address it is 
to specify different core for main.c thread per vpp instance.

What is the “significant problem” you’re running into? Are tests not executing 
in parallel using python multiprocessing, are vpp’s having issues, else? Could 
you describe it a bit more?

-Maciek

[1] https://gerrit.fd.io/r/#/c/13491/
[2] https://git.fd.io/vpp/tree/src/vpp/conf/startup.conf


On 27 Jul 2018, at 17:23, Alec Hothan (ahothan) 
mailto:ahot...@cisco.com>> wrote:

Hi Juraj,
How many instances and what level of performance are you looking at?
Even if you assign different cores to each VPP instance, results can be skewed 
due to interference at the LLC and PCIe/NIC level (this can be somewhat 
mitigated by running on separate sockets)

   Alec


From: mailto:vpp-dev@lists.fd.io>> on behalf of Juraj 
Linkeš mailto:juraj.lin...@pantheon.tech>>
Date: Friday, July 27, 2018 at 7:25 AM
To: "Maciek Konstantynowicz (mkonstan)" 
mailto:mkons...@cisco.com>>
Cc: "vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" 
mailto:vpp-dev@lists.fd.io>>, csit-dev 
mailto:csit-...@lists.fd.io>>
Subject: Re: [vpp-dev] Parallel test execution in VPP Test Framework

Hi Maciek and vpp-devs,

I've run into a significant problem regarding VPP assignment to cores. All VPPs 
that are spawned are assigned to core 1. I looked at 
https://wiki.fd.io/view/VPP/Command-line_Arguments and I guess it's because 
that's the default behavior of VPP (dpdk coremask is not configured and  Note 
that the "main" thread always occupies the lowest core-id specified in the DPDK 
[process-level] coremask.").

Is my reading of the config options accurate?

Obviously, all VPP instances running on the same core goes against running the 
tests on multiple cores. There are a couple of solutions that come to mind:
• Assign VPP instances to cores manually. With possible multiple jobs 
running on a given host, this creates a situation where the different jobs 
don't know cores are already occupied (and by how many VPP instances) and thus 
introduces additional challenges to solve.
• Add an option to override this default behavior and let the Linux CFS 
scheduler assign VPPs to cores or something similar where VP

Re: [vpp-dev] Parallel test execution in VPP Test Framework

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Alec, This is about make test and not real packet forwarding. Per Juraj’s patch 
[1]

Juraj, My understanding is that if you’re starting VPP without specifying core 
placement in startup.conf [2] cpu {..}, then Linux CFS will be placing the 
threads onto available cpu core resources. If you’re saying this is not the 
case, and indeed the wiki comment indicates this, then the way to address it is 
to specify different core for main.c thread per vpp instance.

What is the “significant problem” you’re running into? Are tests not executing 
in parallel using python multiprocessing, are vpp’s having issues, else? Could 
you describe it a bit more?

-Maciek

[1] https://gerrit.fd.io/r/#/c/13491/
[2] https://git.fd.io/vpp/tree/src/vpp/conf/startup.conf

On 27 Jul 2018, at 17:23, Alec Hothan (ahothan) 
mailto:ahot...@cisco.com>> wrote:

Hi Juraj,
How many instances and what level of performance are you looking at?
Even if you assign different cores to each VPP instance, results can be skewed 
due to interference at the LLC and PCIe/NIC level (this can be somewhat 
mitigated by running on separate sockets)

   Alec


From: mailto:vpp-dev@lists.fd.io>> on behalf of Juraj 
Linkeš mailto:juraj.lin...@pantheon.tech>>
Date: Friday, July 27, 2018 at 7:25 AM
To: "Maciek Konstantynowicz (mkonstan)" 
mailto:mkons...@cisco.com>>
Cc: "vpp-dev@lists.fd.io" 
mailto:vpp-dev@lists.fd.io>>, csit-dev 
mailto:csit-...@lists.fd.io>>
Subject: Re: [vpp-dev] Parallel test execution in VPP Test Framework

Hi Maciek and vpp-devs,

I've run into a significant problem regarding VPP assignment to cores. All VPPs 
that are spawned are assigned to core 1. I looked at 
https://wiki.fd.io/view/VPP/Command-line_Arguments and I guess it's because 
that's the default behavior of VPP (dpdk coremask is not configured and  Note 
that the "main" thread always occupies the lowest core-id specified in the DPDK 
[process-level] coremask.").

Is my reading of the config options accurate?

Obviously, all VPP instances running on the same core goes against running the 
tests on multiple cores. There are a couple of solutions that come to mind:
• Assign VPP instances to cores manually. With possible multiple jobs 
running on a given host, this creates a situation where the different jobs 
don't know cores are already occupied (and by how many VPP instances) and thus 
introduces additional challenges to solve.
• Add an option to override this default behavior and let the Linux CFS 
scheduler assign VPPs to cores or something similar where VPPs would land on 
different cores.

Is there some other solution?

Vpp-devs, what do you think about the second solution? What it be possible?

Thanks,
Juraj

From: Maciek Konstantynowicz (mkonstan) [mailto:mkons...@cisco.com]
Sent: Wednesday, July 25, 2018 1:10 PM
To: Juraj Linkeš mailto:juraj.lin...@pantheon.tech>>
Cc: vpp-dev@lists.fd.io; csit-dev 
mailto:csit-...@lists.fd.io>>
Subject: Re: [vpp-dev] Parallel test execution in VPP Test Framework





On 19 Jul 2018, at 15:44, Juraj Linkeš 
mailto:juraj.lin...@pantheon.tech>> wrote:

Hi VPP devs,

I'm implementing parallel test execution of tests in VPP Test Framework (the 
patch is here https://gerrit.fd.io/r/#/c/13491/) and the last big outstanding 
question is how scalable the parallelization actually is.

That’s a good question. What do the tests say? :)



The tests are spawning one VPP instance per each VPPTestCase class

How many VPP instances are spawned and run in parallel? Cause assuming
there is at least one VPPTestCase class per test_, that’s 70 VPP
instances ..




and the question is - how do the required compute resources per each VPP 
instance (cpu, ram, shm) scale and how much resources do we need with 
increasing number of VPP instances running in parallel (in the context of VPP 
Test Framework tests)?

I guess this will vary between tests too. FIB scale tests will require
more fib heap memory. What do the tests say? :)

-Maciek




The second question would be a generic "is there anything else I need to be 
aware of when trying to run VPPs in parallel?".

Thanks,
Juraj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9886): https://lists.fd.io/g/vpp-dev/message/9886
Mute This Topic: https://lists.fd.io/mt/23744978/675185
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9952): https://lists.fd.io/g/vpp-dev/message/9952
Mute This Topic: https://lists.fd.io/mt/23744978/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT VIRL P0 migration to make test completed

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Just wanted to confirm that the last patches for ecmp ip4 and ip6 tests
got merged [1]. With this VIRL P0 [2] test migration to VPP make test is
completed.

Ed, per discussion on last Tue vpp-dev call, you may want to disable
vpp-csit-verify-virl-master [3] voting on VPP patches.

-Maciek

[1] https://gerrit.fd.io/r/#/c/13466/
[2] 
https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit#gid=116058731
[3] https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-virl-master/-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9948): https://lists.fd.io/g/vpp-dev/message/9948
Mute This Topic: https://lists.fd.io/mt/23830748/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Parallel test execution in VPP Test Framework

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

On 19 Jul 2018, at 15:44, Juraj Linkeš 
mailto:juraj.lin...@pantheon.tech>> wrote:

Hi VPP devs,

I'm implementing parallel test execution of tests in VPP Test Framework (the 
patch is here https://gerrit.fd.io/r/#/c/13491/) and the last big outstanding 
question is how scalable the parallelization actually is.

That’s a good question. What do the tests say? :)

The tests are spawning one VPP instance per each VPPTestCase class

How many VPP instances are spawned and run in parallel? Cause assuming
there is at least one VPPTestCase class per test_, that’s 70 VPP
instances ..


and the question is - how do the required compute resources per each VPP 
instance (cpu, ram, shm) scale and how much resources do we need with 
increasing number of VPP instances running in parallel (in the context of VPP 
Test Framework tests)?

I guess this will vary between tests too. FIB scale tests will require
more fib heap memory. What do the tests say? :)

-Maciek


The second question would be a generic "is there anything else I need to be 
aware of when trying to run VPPs in parallel?".

Thanks,
Juraj
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9886): https://lists.fd.io/g/vpp-dev/message/9886
Mute This Topic: https://lists.fd.io/mt/23744978/675185
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[mkons...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9927): https://lists.fd.io/g/vpp-dev/message/9927
Mute This Topic: https://lists.fd.io/mt/23744978/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP l2 path regressions in trending

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Full set of detected regressions is listed in VPP Performance Dashboard [1].

Subset of tests reporting packet throughput regressions ~10%:

1. v18.11-rc0-13 to rc0-19
  a. some tests with 64B throughput drop:
- -10% - 10ge2p1x710-64b-1t1c-eth-l2xcbase-eth-2memif-1lxc-mrr 
- -8% - 10ge2p1x710-64b-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr
- -6% - 10ge2p1x520-imix-1t1c-ethip6srhip6-ip6base-srv6proxy-stat-mrr
  b. Last good: v18.11-rc0-13-g882fcfe
  c. First bad: v18.11-rc0-19-g6a5adc3
  d. git log 882fcfe..6a5adc3 --abbrev-commit --pretty=oneline
6a5adc3 Add config option to use dlmalloc instead of mheap
2a3fb1a make_test: dhcp - add lease time test and wrong XID in OFFER test
8aedf5e vpp-api: improve naming for easier debugging
54d0ac3 VPP crash when run "lb set interface nat4 in " VPP-1343
d6dfd00 Fix GTP-U length header field in encoding
96cc67d Fix GTP-U header in 4x IPv4 vector encapsulation

-Maciek

[1] https://docs.fd.io/csit/master/trending/introduction/index.html
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9921): https://lists.fd.io/g/vpp-dev/message/9921
Mute This Topic: https://lists.fd.io/mt/23804203/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT_VPP_trending_failures: DPDK link bonding

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Summary: https://docs.fd.io/csit/master/trending/introduction/failures.html

CSIT_VPP_trending_failures: DPDK link bonding

Failed_tests_count: 14

Failed_tests_list:
-
  
Failures  Last Failure  Last FailureLast Failure
Test Case 
[#]   [Time][VPP-Build-Id]  [CSIT-Job-Build-Id]
-
10ge2p1x710-imix-4t4c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-imix-4t4c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr   
1220180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-imix-4t4c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 6 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-imix-4t4c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr   4 
20180624 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-175
10ge2p1x710-imix-2t2c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr   2 
20180626 02:07v18.07-rc0~405-g0131b6c~b4518   mrr-daily-build-178
10ge2p1x710-imix-2t2c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 2 
20180626 02:07v18.07-rc0~405-g0131b6c~b4518   mrr-daily-build-178
10ge2p1x710-64b-4t4c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr2 
20180626 02:07v18.07-rc0~405-g0131b6c~b4518   mrr-daily-build-178
10ge2p1x710-imix-1t1c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr   1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-imix-1t1c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr 1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-1t1c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr  1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-1t1c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-4t4c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr  1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-2t2c-1lbdpdk-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-2t2c-1lbdpdk-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr  1 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9725): https://lists.fd.io/g/vpp-dev/message/9725
Mute This Topic: https://lists.fd.io/mt/22724273/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT_VPP_trending_regressions: 1t1c summary

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Summary: 
https://docs.fd.io/csit/master/trending/introduction/index.html#vpp-worker-on-1t1c

Regression_tests_count: 15

Regression_tests_list:

  
Trend   Short-TermLong-Term Regressions   Progressions 
Test Case 
[Mpps]  Change [%]Change [%][#]   [#]

40ge2p1xl710-64b-1t1c-ethip4ipsecbasetnl-ip4base-int-cbc-sha1-mrr 
1.19-82.49-82.492 1
40ge2p1xl710-64b-1t1c-ethip4ipsecscale1000tnl-ip4base-int-cbc-sha1-mrr
0.55-80.77-80.772 1
40ge2p1xl710-64b-1t1c-ethip4ipsecbasetnlsw-ip4base-int-cbc-sha1-mrr   
1.07-66.01-68.272 1
10ge2p1x520-64b-1t1c-eth-l2xcbase-testpmd-mrr 
23.72   -8.48 -8.48 1 0
40ge2p1xl710-imix-1t1c-eth-l2xcbase-eth-2memif-1lxc-mrr   
4.47-6.27 -11.181 0
40ge2p1xl710-imix-1t1c-eth-l2bdbasemaclrn-eth-2memif-1lxc-mrr 
3.85-6.21 -11.161 0
10ge2p1x520-64b-1t1c-ethip4-ip4base-l3fwd-mrr 
24.11   -5.26 -5.26 1 0
40ge2p1xl710-64b-1t1c-eth-l2xcbase-eth-2memif-1lxc-mrr
5.77-4.88 -4.88 1 0
10ge2p1x710-64b-1t1c-eth-l2bdbasemaclrn-iacl50-stateless-flows10k-mrr 
4.26-3.15 -4.84 1 0
10ge2p1x520-64b-1t1c-eth-l2bdbasemaclrn-iacl50-stateless-flows10k-mrr 
4.29-2.8  -4.64 1 0
10ge2p1x710-64b-1t1c-eth-l2bdbasemaclrn-iacl1-stateless-flows10k-mrr  
4.28-2.55 -4.34 1 0
10ge2p1x520-64b-1t1c-eth-l2bdbasemaclrn-iacl1-stateless-flows10k-mrr  
4.31-2.29 -4.25 1 0
10ge2p1x520-imix-1t1c-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr 
3.29-2.0  -2.0  1 0
10ge2p1x710-imix-1t1c-eth-l2xcbase-testpmd-mrr
6.69-0.0  -0.0  1 0
40ge2p1xl710-64b-1t1c-ethip4ipsecbasetnl-ip4base-int-aes-gcm-mrr  
6.530.0   -8.81 1 0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9726): https://lists.fd.io/g/vpp-dev/message/9726
Mute This Topic: https://lists.fd.io/mt/22724276/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT_VPP_trending_failures: ip4scale, ip6scale

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Summary: https://docs.fd.io/csit/master/trending/introduction/failures.html

CSIT_VPP_trending_failures: DPDK link bonding

Failed_tests_count: 18

Failed_tests_list:
-
  
Failures  Last Failure  Last FailureLast Failure
Test Case 
[#]   [Time][VPP-Build-Id]  [CSIT-Job-Build-Id]
-
10ge2p1x520-78b-4t4c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-78b-1t1c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-78b-4t4c-ethip6-ip6base-ip6scale200k-mrr  
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-78b-2t2c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-64b-2t2c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-78b-1t1c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-78b-4t4c-ethip6-ip6base-ip6scale200k-mrr  
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-64b-1t1c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-64b-4t4c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-64b-4t4c-ethip4-ip4scale200k-mrr  
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-64b-2t2c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-78b-2t2c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-64b-1t1c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-64b-4t4c-ethip4-ip4scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-78b-4t4c-ethip6-ip6base-ip6scale2m-mrr
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x710-64b-4t4c-ethip4-ip4scale200k-mrr  
1420180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
10ge2p1x520-64b-2t2c-ethip4-ip4scale200k-mrr  2 
20180623 16:04v18.07-rc0~394-ge0d2bd6~b4508   mrr-daily-build-173
10ge2p1x710-64b-2t2c-ethip4-ip4scale200k-mrr  1 
20180621 16:06v18.07-rc0~384-g85aa490~b4489   mrr-daily-build-169
--=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9724): https://lists.fd.io/g/vpp-dev/message/9724
Mute This Topic: https://lists.fd.io/mt/22724269/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] CSIT_VPP_trending_failures: IPSec

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Summary: https://docs.fd.io/csit/master/trending/introduction/failures.html

CSIT_VPP_trending_failures: DPDK link bonding

Failed_tests_count: 10

Failed_tests_list:
-
  
Failures  Last Failure  Last FailureLast Failure
Test Case 
[#]   [Time][VPP-Build-Id]  [CSIT-Job-Build-Id]
-
40ge2p1xl710-64b-1t1c-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-mrr 5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-4t4c-ethip4ipsecscale1000tnl-ip4base-int-cbc-sha1-mrr5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-1t1c-ethip4ipsecbasetnl-ip4base-int-aes-gcm-mrr  5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-2t2c-ethip4ipsecbasetnl-ip4base-int-aes-gcm-mrr  5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-4t4c-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-mrr 5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-1t1c-ethip4ipsecbasetnlsw-ip4base-int-aes-gcm-mrr5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-2t2c-ethip4ipsecscale1000tnl-ip4base-int-aes-gcm-mrr 5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-4t4c-ethip4ipsecbasetnlsw-ip4base-int-aes-gcm-mrr5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-2t2c-ethip4ipsecbasetnlsw-ip4base-int-aes-gcm-mrr5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
40ge2p1xl710-64b-4t4c-ethip4ipsecbasetnl-ip4base-int-aes-gcm-mrr  5 
20180626 16:10v18.07-rc0~410-g8d2e9bd~b4524   mrr-daily-build-179
--=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9723): https://lists.fd.io/g/vpp-dev/message/9723
Mute This Topic: https://lists.fd.io/mt/22724267/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [csit-dev] [vpp-dev] VPP make test gives false positive

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Any updates on the latest status?

-Maciek

> On 22 Jun 2018, at 18:07, Ole Troan  wrote:
> 
>> 13188 is already merged.
> 
> 13186 fails consistently locally too, investigating.
> I think this was introduced with a98346f664aae148d26a8e158008b773d73db96f
> 
> Cheers,
> Ole
> 
> 
>> 
>> From: vpp-dev@lists.fd.io  On Behalf Of Ed Kern via 
>> Lists.Fd.Io
>> Sent: Friday, June 22, 2018 12:07 PM
>> To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) 
>> 
>> Cc: vpp-dev@lists.fd.io
>> Subject: Re: [csit-dev] [vpp-dev] VPP make test gives false positive
>> Importance: High
>> 
>> Hey,
>> 
>> responding because I dont see one from klement.
>> 
>> looks like he has two gerrits pending
>> 
>> https://gerrit.fd.io/r/#/c/13188/
>> 
>> to fix broken interfaces.. (this passed verification so trying to get this 
>> one merged)
>> 
>> and
>> 
>> https://gerrit.fd.io/r/#/c/13186/
>> 
>> for the retries in error..this is not passing verification but i have my 
>> fingers crossed that it will once
>> 13188 is merged.
>> 
>> Hoping to get this behind us before the weekend..
>> 
>> Ed
>> 
>> 
>> 
>> 
>> On Jun 22, 2018, at 3:03 AM, Jan Gelety via Lists.Fd.Io 
>>  wrote:
>> 
>> Hello,
>> 
>> VPP make test gives false positive results at the moment - there are cca 110 
>> tests failed but build is marked successful.
>> 
>> The last correct run on ubuntu is [1].
>> 
>> The first false positive run on ubutnu is [2] – trigerred by Florin’s patch 
>> [3].
>> 
>> It seems that the first failing test is test_ip4_irb.TestIpIrb:
>> 
>> 00:22:48.177 
>> ==
>> 00:22:53.685 VAPI test
>> 00:22:53.685 
>> ==
>> 00:22:53.685 run C VAPI tests
>>  SKIP
>> 00:22:53.685 run C++ VAPI tests  
>>  SKIP
>> 00:22:53.685
>> 00:22:53.685 
>> ==
>> 00:22:53.685 ERROR: setUpClass (test_ip4_irb.TestIpIrb)
>> 00:22:53.685 
>> --
>> 00:22:53.685 Traceback (most recent call last):
>> 00:22:53.685   File 
>> "/w/workspace/vpp-verify-master-ubuntu1604/test/test_ip4_irb.py", line 57, 
>> in setUpClass
>> 00:22:53.685 cls.create_loopback_interfaces(range(1))
>> 00:22:53.685   File 
>> "/w/workspace/vpp-verify-master-ubuntu1604/test/framework.py", line 571, in 
>> create_loopback_interfaces
>> 00:22:53.685 setattr(cls, intf.name, intf)
>> 00:22:53.685   File 
>> "/w/workspace/vpp-verify-master-ubuntu1604/test/vpp_interface.py", line 91, 
>> in name
>> 00:22:53.685 return self._name
>> 00:22:53.685 AttributeError: 'VppLoInterface' object has no attribute '_name'
>> 00:22:53.685
>> 
>> And all following tests failes with connection failur during test case setup:
>> 
>> 00:22:53.689 
>> ==
>> 00:22:53.689 ERROR: setUpClass (test_dvr.TestDVR)
>> 00:22:53.689 
>> --
>> 00:22:53.689 Traceback (most recent call last):
>> 00:22:53.689   File 
>> "/w/workspace/vpp-verify-master-ubuntu1604/test/framework.py", line 362, in 
>> setUpClass
>> 00:22:53.689 cls.vapi.connect()
>> 00:22:53.689   File 
>> "/w/workspace/vpp-verify-master-ubuntu1604/test/vpp_papi_provider.py", line 
>> 141, in connect
>> 00:22:53.689 self.vpp.connect(self.name, self.shm_prefix)
>> 00:22:53.689   File "build/bdist.linux-x86_64/egg/vpp_papi.py", line 699, in 
>> connect
>> 00:22:53.689 async)
>> 00:22:53.689   File "build/bdist.linux-x86_64/egg/vpp_papi.py", line 670, in 
>> connect_internal
>> 00:22:53.689 raise IOError(2, 'Connect failed')
>> 00:22:53.689 IOError: [Errno 2] Connect failed
>> 00:22:53.689
>> 
>> But result is OK:
>> 00:22:53.689 Ran 156 tests in 162.407s
>> 00:22:53.689
>> 00:22:53.689 FAILED (errors=114, skipped=118)
>> 00:22:53.691 1 test(s) failed, 3 attempt(s) left
>> 00:22:53.697 Running tests using custom test runner
>> 00:22:53.697 Active filters: file=None, class=None, function=None
>> 00:22:53.697 0 out of 0 tests match specified filters
>> 00:22:53.697 Not running extended tests (some tests will be skipped)
>> 00:22:53.697
>> 00:22:53.697 Ran 0 tests in 0.000s
>> 00:22:53.697
>> 00:22:53.697 OK
>> 00:22:53.697 0 test(s) failed, 2 attempt(s) left
>> 00:22:53.757 Killing possible remaining process IDs:  20845 21195 21197
>> 00:22:53.766 make[2]: Leaving directory 
>> '/w/workspace/vpp-verify-master-ubuntu1604/test'
>> 00:22:53.767 make[1]: Leaving directory 
>> '/w/workspace/vpp-verify-master-ubuntu1604'
>> 00:22:53.768 + '[' x == x1 ']'
>> 00:22:53.768 + echo 
>> '***'
>> 00:22:53.768 
>> 

Re: [vpp-dev] [csit-dev] Draft proposal to introduce a new set of integration tests

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Adding vpp-dev.

Following reviews in csit-dev and CSIT project calls, here is an updated
proposal for a new set of FD.io<http://FD.io> VPP integration tests. The idea 
is to,
over time, migrate the VIRL test cases to VPP_Path and VPP_Device
integration tests describe below.

Test coverage gaps between existing VIRL and VPP make_test tests have been
documented in an online sheet [1]. Many thanks to Jan Gelety for the
analysis and CSIT contributors for their reviews.

Unless issues/objections, plan is to progress this work during rls1807 dev
cycle.

Hope this makes sense, let us know if questions.

-Maciek
(CSIT PTL)

[1] vpp-csit-verify-virl-test-gaps, 
https://docs.google.com/spreadsheets/d/1PciV8XN9v1qHbIRUpFJoqyES29_vik7lcFDl73G1usc/edit?usp=sharing



VPP_Path Integration Tests
--

* Objective:(Integration) Functional acceptance of VPP feature
paths driven by use cases.
. Use cases defined as combinations of configurations
  and offered loads invoking multiple VPP packet
  processing paths.
. Tester: "Does it do what is expected based on
  configuration and offered load?"
. Includes regression tests based on field found bugs.
* Environment:  VPP make_test.
. VPP test framework based on Python unittest.
* Testing:  VPP SW only.
. api: vpp-papi.
. dp: pg-rx-{vpp_graph_nodes_path}–tx-pg.
* Results:  Test [PASS|FAIL].
* Analytics:gcc gcov for code line coverage.
* Continuous:   FD.io<http://FD.io> per vpp patch execution with jjb verify 
voting.
. Separate job from current vpp make verify(?).
* Availability: New development in FD.io<http://FD.io> CSIT.
. Partial coverage in VPP_Multi_Node Tests.
* Project:  FD.io<http://FD.io> CSIT.
. Plan: introduce in rls1807, test coverage driven by
  priority use cases and VIRL equivalence.
. Future:
.. Negative tests: API fuzzing (/Fuzzapi/API-fuzzer),
   packet/protocol fuzzing (Cisco-Talos/mutiny-fuzzer).

VPP_Device Integration Tests


* Objective:(Integration) Functional acceptance of VPP device driver
interactions.
. Tester: "Known VPP device and feature interactions
  must work!"
* Environment:  FD.io<http://FD.io> CSIT-CPL labs.
. CSIT Python and Robot Framework libraries and tests.
* Testing:  VPP SW on compute HW with network devices.
. VPP SW:
.. api: vpp-papi.
.. dp: int-dev–rx-{simple_VPP_graph_nodes_path}-tx-dev-int.
. Compute HW:
.. Xeon-Skylake, Arm and Atom nodes.
.. Cisco, Intel, Mellanox NICs.
* Results:  Test [PASS|FAIL].
* Analytics:None.
* Continuous:   FD.io<http://FD.io> per vpp patch execution with jjb verify 
voting.
* Availability: Depends on server and processor types.
. Xeon:
.. rls1807 - initial VPP and Scapy Pod/Container POC
   with VF drivers.
.. Future - per vpp patch execution with jjb verify
   voting.
. Arm:
.. rls1807 - initial VPP and Scapy Pod/Container POC
   with VF drivers.
.. Future - per vpp patch execution with jjb verify
   voting.
. Atom: TBC.
* Project:  FD.io<http://FD.io> CSIT.
    . Plan: progress in rls1807.


On 6 Jun 2018, at 14:25, Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io 
mailto:mkonstan=cisco@lists.fd.io>> wrote:

Thanks for all comments!

Here is an updated summary of both new test categories based
on received feedback.

(I removed the dependencies for clarity).

I’m also sending a separate email with a straw-man proposal
for parallelising functional HW device tests in VPP_Device.



VPP_Path Integration Tests
--

* Objective:(Integration) Functional acceptance of VPP feature
paths driven by use cases (combinations of multiple
nodes, configurations and offered loads).
. Tester: "Must not break known node and feature
  interactions!"
. Includes regression tests based on field reported bugs.
* Environment:  VPP make_test.
* Testing:  VPP SW only.
. api: vpp-papi.
. dp: pg-rx-{vpp_graph_nodes_path}–tx-pg.
* Results:  Test [PASS|FAIL].
* Analytics:gcc gcov for code line coverage.
* Continuous:   FD.io<http://fd.io/> per vpp patch execution with jjb verify 
voting.
. Separate job from current vpp make verify.
* Availability: New development in FD.io<htt

Re: [vpp-dev] How to trigger perf test and compare the results

[Maciek Konstantynowicz (mkonstan) via Lists.Fd.Io]

Hi Zhiyong,

We’re in the process of adding gerrit triggers
to provide more complete coverage of MRR tests
per vpp patch.

What (NIC,featureset) combination(s) are you after?

-Maciek

On 31 May 2018, at 10:55, Damjan Marion 
mailto:dmarion.li...@gmail.com>> wrote:


Dear Zhiyong,

csit-dev is mailing list of csit folks, who might help with questions I did not 
answer
.
--
Damjan

On 31 May 2018, at 07:04, Yang, Zhiyong 
mailto:zhiyong.y...@intel.com>> wrote:

Hi damjan,

I try to post commit “csit-dev”, however nothing happens, 
what’s wrong with me?

Thanks
Zhiyong

From: Damjan Marion [mailto:dmarion.li...@gmail.com]
Sent: Wednesday, May 30, 2018 5:52 PM
To: Yang, Zhiyong mailto:zhiyong.y...@intel.com>>
Cc: vpp-dev@lists.fd.io; Kinsella, Ray 
mailto:ray.kinse...@intel.com>>; csit-dev 
mailto:csit-...@lists.fd.io>>
Subject: Re: How to trigger perf test and compare the results

+csit-dev

You can compare with numbers available in perf dashboard.

https://docs.fd.io/csit/master/trending/introduction/index.html

--
Damjan


On 29 May 2018, at 11:17, Yang, Zhiyong 
mailto:zhiyong.y...@intel.com>> wrote:

Hi Guys,

   I need CSIT perf testing  to test patch.
   I have known that vpp-verify-perf-l2 can trigger perf test, but I 
only see the X520 results.

My questions are :

1.  how to trigger XL710 perf test?
2.  What do we compare it to … have the nightly build results?  where do I 
find those?
3.  These are MRR (maximum receive rate tests), how do we trigger NDP/PDR?


Thanks
Zhiyong