On 7 December 2017 at 17:01, Bogdan Pricope <bogdan.pric...@linaro.org> wrote:
> TX is at line rate. Probably will get RX at line rate in direct mode, too. > Problem is how can you see the performance increase/degradation if you > can process more than line rate with one core? > > I guess .. enable csum option... ? > > one question when people download odp, run and see line rate and happy and other is what we will use in tests. We can check that line rate is till line rate and scheduler on both ends is some value bellow line rate. > On 7 December 2017 at 15:46, Maxim Uvarov <maxim.uva...@linaro.org> wrote: > > nice. TX is on line rate, right? Next step probably to add RX path > without > > scheduler. And we will have good testing environment. > > > > > > On 7 December 2017 at 16:12, Bogdan Pricope <bogdan.pric...@linaro.org> > > wrote: > >> > >> More results with odp_generator in lava setup: > >> > >> 7.6 mpps (TX) / 5.9 mpps (RX) - api-next with PR313 (Petri): > >> 8.3 mpps (TX) / 6.3 mpps (RX) - api-next with PR313 (Petri) + > >> remove 1m sleep + replace atomic counters > >> 14.8 mpps (TX) / 6.5 mpps (RX) - api-next with PR313 (Petri) + remove > >> 1m sleep + replace atomic counters + remove csum > >> calculation/validation > >> 14.8 mpps (TX) / 6.8 mpps (RX) - master with PR327 (remove 1m sleep + > >> replace atomic counters + remove csum calculation/validation) > >> > >> /Bogdan > >> > >> > >> On 6 December 2017 at 13:49, Maxim Uvarov <maxim.uva...@linaro.org> > wrote: > >> > small update. Double checked that increasing num of desc does not give > >> > any > >> > effect in odp_generator. > >> > > >> > Disable check sums in odp_generator increases TX from 7M to 13M pps > and > >> > RX > >> > from 5.9M to 6.1M pps. > >> > Because of generator uses predefined packets with calculated checksum > - > >> > there is no need to enable checksum inside generator. > >> > > >> > It looks like problem inside DPDK driver itself. > >> > > >> > For this PR I think we need to merge it together with changes to > >> > odp_generator (the same as for l2fwd) to enable hw check sum, > >> > which has to be disabled by default. > >> > > >> > Maxim. > >> > > >> > > >> > On 6 December 2017 at 10:46, Maxim Uvarov <maxim.uva...@linaro.org> > >> > wrote: > >> >> > >> >> skip this message. I will recheck. Pushed to lava wrong branch. > >> >> > >> >> On 6 December 2017 at 10:42, Maxim Uvarov <maxim.uva...@linaro.org> > >> >> wrote: > >> >>> > >> >>> Ilias was right yesterday. If number of descriptors increased to > 1024 > >> >>> then TX became again 10M. > >> >>> > >> >>> + ret = rte_eth_tx_queue_setup(port_id, i, > >> >>> + > >> >>> dev_info.tx_desc_lim.nb_max > >> >>> > 1024 ? 1024 : dev_info.tx_desc_lim.nb_max, > >> >>> > >> >>> rte_eth_dev_socket_id(port_id), > >> >>> txconf); > >> >>> > >> >>> + ret = rte_eth_rx_queue_setup(port_id, i, > >> >>> + > >> >>> dev_info.rx_desc_lim.nb_max > >> >>> > 1024 ? 1024 : dev_info.rx_desc_lim.nb_max, > >> >>> > >> >>> rte_eth_dev_socket_id(port_id), > >> >>> NULL, > >> >>> pkt_dpdk->pkt_pool); > >> >>> > >> >>> > >> >>> > >> >>> > >> >>> Maxim. > >> >>> > >> >>> On 5 December 2017 at 11:20, Elo, Matias (Nokia - FI/Espoo) > >> >>> <matias....@nokia.com> wrote: > >> >>>> > >> >>>> When I tested enabling HW checksum with Fortville NICs (i40e) the > >> >>>> slower > >> >>>> driver path alone caused ~20% throughput drop on l2fwd test. This > was > >> >>>> without actually calculating the checksums, I simply forced the > >> >>>> slower > >> >>>> driver path (no vectorization). > >> >>>> > >> >>>> -Matias > >> >>>> > >> >>>> > >> >>>> > On 5 Dec 2017, at 8:59, Bogdan Pricope < > bogdan.pric...@linaro.org> > >> >>>> > wrote: > >> >>>> > > >> >>>> > On RX side is kind-of expected result since it uses scheduler > mode. > >> >>>> > > >> >>>> > On TX side there is this drop from 10 mpps to 7.69 mpps that is > >> >>>> > unexpected. > >> >>>> > > >> >>>> > So Petri, when you said: > >> >>>> > "DPDK uses less optimized driver code (on Intel NICs at least) > when > >> >>>> > any of the L4 checksum offloads is enabled." > >> >>>> > > >> >>>> > you were referring to this kind of drop in performance? > >> >>>> > > >> >>>> > There is that 'folklore' that SW csum is faster on small packets > >> >>>> > while > >> >>>> > HW csum is faster on bigger packets. Do you have this kind of > data? > >> >>>> > > >> >>>> > Anyway, for this particular case (odp_generator), since UDP > >> >>>> > header/payload is not changing during the test (for now), csum is > >> >>>> > calculated only once at the beginning of the test: so we are > >> >>>> > comparing > >> >>>> > HW IPv4 + HW UDP csum vs. SW IPv4 csum.... yet, the differences > in > >> >>>> > performance is huge... > >> >>>> > > >> >>>> > > >> >>>> > On 4 December 2017 at 20:37, Maxim Uvarov < > maxim.uva...@linaro.org> > >> >>>> > wrote: > >> >>>> >> I added isocpus and mounted huge page TX became more stable at > >> >>>> >> 7.6M. > >> >>>> >> But > >> >>>> >> anyway it's better to test performance for this PR because > >> >>>> >> previous > >> >>>> >> speed was 10M. > >> >>>> >> > >> >>>> >> Maxim. > >> >>>> >> > >> >>>> >> On 12/04/17 19:42, Honnappa Nagarahalli wrote: > >> >>>> >>> Can you run with Linux-DPDK in ODP 2.0? > >> >>>> >>> > >> >>>> >>> On 4 December 2017 at 09:54, Maxim Uvarov > >> >>>> >>> <maxim.uva...@linaro.org> > >> >>>> >>> wrote: > >> >>>> >>>> after clean patches apply and fix in run scripts I made it > run. > >> >>>> >>>> > >> >>>> >>>> But results is really bad. --enable-dpdk-zero-copy > >> >>>> >>>> > >> >>>> >>>> TX rate is: > >> >>>> >>>> 7673155 pps > >> >>>> >>>> > >> >>>> >>>> RX rate is: > >> >>>> >>>> 5989846 pps > >> >>>> >>>> > >> >>>> >>>> > >> >>>> >>>> Before patch PR 313 TX was 10M pps. > >> >>>> >>>> > >> >>>> >>>> I re run task and TX is 3.3M pps. All tests are single core. > So > >> >>>> >>>> something strange happens in lava or this PR. > >> >>>> >>>> > >> >>>> >>>> Maxim. > >> >>>> >>>> > >> >>>> >>>> > >> >>>> >>>> On 12/04/17 17:03, Bogdan Pricope wrote: > >> >>>> >>>>> On TX (https://lng.validation.linaro.org/scheduler/job/ > 23252.0) > >> >>>> >>>>> I > >> >>>> >>>>> see: > >> >>>> >>>>> > >> >>>> >>>>> ODP_REPO='https://github.com/muvarov/odp' > >> >>>> >>>>> ODP_BRANCH='api-next' > >> >>>> >>>>> > >> >>>> >>>>> > >> >>>> >>>>> On RX (https://lng.validation.linaro.org/scheduler/job/ > 23252.1) > >> >>>> >>>>> I > >> >>>> >>>>> see: > >> >>>> >>>>> > >> >>>> >>>>> ODP_REPO='https://github.com/muvarov/odp' > >> >>>> >>>>> ODP_BRANCH='devel/api-next_shsum' > >> >>>> >>>>> > >> >>>> >>>>> > >> >>>> >>>>> or are you referring to other test? > >> >>>> >>>>> > >> >>>> >>>>> > >> >>>> >>>>> On 4 December 2017 at 15:53, Maxim Uvarov > >> >>>> >>>>> <maxim.uva...@linaro.org> wrote: > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> On 4 December 2017 at 15:11, Bogdan Pricope > >> >>>> >>>>>> <bogdan.pric...@linaro.org> > >> >>>> >>>>>> wrote: > >> >>>> >>>>>>> > >> >>>> >>>>>>> You need to put 313 on TX side (not RX). > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> both rx and tx have patches from 313. l2fwd works on recv > >> >>>> >>>>>> side. > >> >>>> >>>>>> Generator > >> >>>> >>>>>> does not work. > >> >>>> >>>>>> > >> >>>> >>>>>> Maxim. > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>>> > >> >>>> >>>>>>> > >> >>>> >>>>>>> On 4 December 2017 at 13:19, Savolainen, Petri (Nokia - > >> >>>> >>>>>>> FI/Espoo) > >> >>>> >>>>>>> <petri.savolai...@nokia.com> wrote: > >> >>>> >>>>>>>> Is the DPDK version 17.08 ? Other versions might not work > >> >>>> >>>>>>>> properly. > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> -Petri > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> From: Maxim Uvarov [mailto:maxim.uva...@linaro.org] > >> >>>> >>>>>>>> Sent: Monday, December 04, 2017 1:10 PM > >> >>>> >>>>>>>> To: Savolainen, Petri (Nokia - FI/Espoo) > >> >>>> >>>>>>>> <petri.savolai...@nokia.com> > >> >>>> >>>>>>>> Cc: Bogdan Pricope <bogdan.pric...@linaro.org>; > >> >>>> >>>>>>>> lng-odp-forward > >> >>>> >>>>>>>> <lng-odp@lists.linaro.org> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> Subject: Re: [lng-odp] odp dpdk > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> 313 does not work also: > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> https://lng.validation.linaro.org/scheduler/job/23242.1 > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> I will replace RX side to l2fwd and see that will be > there. > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> Maxim. > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> On 4 December 2017 at 13:46, Savolainen, Petri (Nokia - > >> >>>> >>>>>>>> FI/Espoo) > >> >>>> >>>>>>>> <petri.savolai...@nokia.com> wrote: > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> Maxim, try https://github.com/Linaro/odp/pull/313 It has > >> >>>> >>>>>>>> been > >> >>>> >>>>>>>> tested to > >> >>>> >>>>>>>> fix > >> >>>> >>>>>>>> checksum insert for 10/40GE Intel NICs. > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> -Petri > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>>>>> -----Original Message----- > >> >>>> >>>>>>>>> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] > On > >> >>>> >>>>>>>>> Behalf Of > >> >>>> >>>>>>>>> Bogdan Pricope > >> >>>> >>>>>>>>> Sent: Monday, December 04, 2017 12:21 PM > >> >>>> >>>>>>>>> To: Maxim Uvarov <maxim.uva...@linaro.org> > >> >>>> >>>>>>>>> Cc: lng-odp-forward <lng-odp@lists.linaro.org> > >> >>>> >>>>>>>>> Subject: Re: [lng-odp] odp dpdk > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> I suspect this is actually caused by csum issue in TX > side: > >> >>>> >>>>>>>>> on > >> >>>> >>>>>>>>> RX, > >> >>>> >>>>>>>>> socket pktio does not validate csum (and accept the > >> >>>> >>>>>>>>> packets) > >> >>>> >>>>>>>>> but on > >> >>>> >>>>>>>>> dpdk pktio the csum is validated and packets are dropped. > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> I am not seeing this in my setup because default > txq_flags > >> >>>> >>>>>>>>> for > >> >>>> >>>>>>>>> igb > >> >>>> >>>>>>>>> driver (1G interface) is > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> .txq_flags = 0 > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> while for ixgbe (10G interface) is: > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> .txq_flags = ETH_TXQ_FLAGS_NOMULTSEGS | > >> >>>> >>>>>>>>> ETH_TXQ_FLAGS_NOOFFLOADS, > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> /B > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> > >> >>>> >>>>>>>>> On 1 December 2017 at 23:47, Maxim Uvarov > >> >>>> >>>>>>>>> <maxim.uva...@linaro.org> > >> >>>> >>>>>>>>> wrote: > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> Looking to dpdk pktio support and generator. It looks > like > >> >>>> >>>>>>>>>> receive > >> >>>> >>>>>>>>>> part > >> >>>> >>>>>>>>>> is broken. If for receive I use sockets it works well > but > >> >>>> >>>>>>>>>> receive > >> >>>> >>>>>>>>>> with > >> >>>> >>>>>>>>>> dpdk does not get any packets. For both master and > >> >>>> >>>>>>>>>> api-next. > >> >>>> >>>>>>>>>> Can > >> >>>> >>>>>>>>>> somebody confirm please that it's so. Lava is not supper > >> >>>> >>>>>>>>>> friendly to > >> >>>> >>>>>>>>>> debug issue. > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> 1. Recv > >> >>>> >>>>>>>>>> odp_generator -I 0 -m r -c 0x4 > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> https://lng.validation.linaro.org/scheduler/job/23206.1 > >> >>>> >>>>>>>>>> Network devices using DPDK-compatible driver > >> >>>> >>>>>>>>>> ============================================ > >> >>>> >>>>>>>>>> 0000:07:00.1 '82599ES 10-Gigabit SFI/SFP+ Network > >> >>>> >>>>>>>>>> Connection > >> >>>> >>>>>>>>>> 10fb' > >> >>>> >>>>>>>>>> drv=igb_uio unused= > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> 2. Send > >> >>>> >>>>>>>>>> odp_generator -I 0 --srcmac 38:ea:a7:93:98:94 --dstmac > >> >>>> >>>>>>>>>> 38:ea:a7:93:83:a0 > >> >>>> >>>>>>>>>> --srcip 192.168.100.2 --dstip 192.168.100.1 -m u -i 0 -c > >> >>>> >>>>>>>>>> 0x8 > >> >>>> >>>>>>>>>> -p 18 -e > >> >>>> >>>>>>>>>> 5000 -f 5001 -n 800000000 > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> https://lng.validation.linaro.org/scheduler/job/23206.0 > >> >>>> >>>>>>>>>> > >> >>>> >>>>>>>>>> Thank you, > >> >>>> >>>>>>>>>> Maxim. > >> >>>> >>>>>>>> > >> >>>> >>>>>>>> > >> >>>> >>>>>> > >> >>>> >>>>>> > >> >>>> >>>> > >> >>>> >> > >> >>>> > >> >>> > >> >> > >> > > > > > >
