RE: E810 VLAN offload wrong behavior
Hi Ivan, Yesterday's fix you mentioned in your previous message worked very well. Hope it will be included in the next 24.11.3 version. Thank you very much Ernesto -Original Message- From: Ivan Malov Sent: Tuesday, July 15, 2025 10:32 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Sorry, the bug might already exist. See [1]. So may be you can add your findings as a comment there. Thank you. [1] https://bugs.dpdk.org/show_bug.cgi?id=1677 On Tue, 15 Jul 2025, Ernesto Ruffini wrote: > Hi Ivan, > It worked! > > ICE_INIT: ice_set_rx_function(): Rx Burst Bulk Alloc Preconditions are > satisfied. Rx Burst Bulk Alloc function will be used on port 0. > > And all the flags are there, with both one and four queues. > > Any idea on where to report this as a bug in the AVX2 OFFLOAD Vector > Rx driver function, as apparently it is? > > Thank you > Ernesto > > > -Original Message- > From: Ivan Malov > Sent: Monday, July 14, 2025 18:12 > To: Ernesto Ruffini > Cc: [email protected] > Subject: RE: E810 VLAN offload wrong behavior > > Hi Ernesto, > > That's interesting. To be honest, I'm not an expert in this driver. So > far it does not seem the issue is with the specific Rx function, yet > may be it pays to temporarily change "#ifdef RTE_ARCH_X86" to "#if 0" > on line [1] in the original source of 24.11.2 and rebuild? Just to > make sure the Rx method is a don't care. > > Thank you. > > [1] > https://github.com/DPDK/dpdk/blob/4c5634de9f933555bab9c64a533d3dca9990 > 71cd/d > rivers/net/ice/ice_rxtx.c#L3489 > > On Mon, 14 Jul 2025, Ernesto Ruffini wrote: > >> Hi Ivan, >> Thank you for your quick answer. >> I tried to apply the patch but got a compilation error: >> >> ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer >> to incomplete type ?const struct ci_rx_queue? >> qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, >> rxq->reg_idx); >> >> So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there >> it compiled. >> Unfortunately it got consistent, but worse: >> The selected rx function is the same: >> ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). >> But it now drops the VLAN, without signaling it, in all cases. >> With 1 queue: >> ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD >> RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: >> ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD >> RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD >> >> I double checked with plain 24.11.2 and indeed with just one queue it >> is still >> ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD >> RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED >> RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD >> >> So I'm confident the lab environment is still good >> >> Thank you >> Ernesto >> >> >> -Original Message- >> From: Ivan Malov >> Sent: Monday, July 14, 2025 14:49 >> To: Ernesto Ruffini >> Cc: [email protected] >> Subject: RE: E810 VLAN offload wrong behavior >> >> Hi Ernesto, >> >> On Mon, 14 Jul 2025, Ernesto Ruffini wrote: >> >>> Hi Ivan, >>> Thanks for the hint. >>> I ran testpmd with the suggested parameter, but I cannot find where >>> the function gets selected. >>> I attach the full log here. >> >> Thanks. The message to look for appears closer to the end of start > sequence: >>> ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). >> >> But before investigating the code of that Rx function, perhaps it >> pays to make sure patch [1] is applied. Everything is so recent that >> most likely it has not landed 24.11 release yet. Theoretically, the >> bug the patch is trying to fix has something to do with multi-queue >> and VLAN stripping. See if you can't apply the patch in 24.11 or >> rebuild from the current main of dpdk-next-net repository. >> >> Thank you. >> >> [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html >> >>> Thanks >>> Ernesto >>> >>> -Original Message- >>> From: Ivan Malov >>> Sent: Monday, July 14, 2025 02:04 >>> To: Ernesto Ruffini >>> Cc: [email protected] >>> Subject: Re: E810 VLAN offload wrong behavior >>> >>> Hi Ernesto, >>> >>> On Fri, 11 Jul 2025, Ernesto Ruf
RE: E810 VLAN offload wrong behavior
Sorry, the bug might already exist. See [1]. So may be you can add your findings as a comment there. Thank you. [1] https://bugs.dpdk.org/show_bug.cgi?id=1677 On Tue, 15 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, It worked! ICE_INIT: ice_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port 0. And all the flags are there, with both one and four queues. Any idea on where to report this as a bug in the AVX2 OFFLOAD Vector Rx driver function, as apparently it is? Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 18:12 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, That's interesting. To be honest, I'm not an expert in this driver. So far it does not seem the issue is with the specific Rx function, yet may be it pays to temporarily change "#ifdef RTE_ARCH_X86" to "#if 0" on line [1] in the original source of 24.11.2 and rebuild? Just to make sure the Rx method is a don't care. Thank you. [1] https://github.com/DPDK/dpdk/blob/4c5634de9f933555bab9c64a533d3dca999071cd/d rivers/net/ice/ice_rxtx.c#L3489 On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thank you for your quick answer. I tried to apply the patch but got a compilation error: ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer to incomplete type ?const struct ci_rx_queue? qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, rxq->reg_idx); So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there it compiled. Unfortunately it got consistent, but worse: The selected rx function is the same: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But it now drops the VLAN, without signaling it, in all cases. With 1 queue: ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD I double checked with plain 24.11.2 and indeed with just one queue it is still ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD So I'm confident the lab environment is still good Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 14:49 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thanks for the hint. I ran testpmd with the suggested parameter, but I cannot find where the function gets selected. I attach the full log here. Thanks. The message to look for appears closer to the end of start sequence: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But before investigating the code of that Rx function, perhaps it pays to make sure patch [1] is applied. Everything is so recent that most likely it has not landed 24.11 release yet. Theoretically, the bug the patch is trying to fix has something to do with multi-queue and VLAN stripping. See if you can't apply the patch in 24.11 or rebuild from the current main of dpdk-next-net repository. Thank you. [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html Thanks Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 02:04 To: Ernesto Ruffini Cc: [email protected] Subject: Re: E810 VLAN offload wrong behavior Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: Hi, We found a strange behavior of the E810 VLAN offload. We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and COMMS DDP 1.3.46 The NIC receives an IPv4/UDP packet inside VLAN 300. If we run dpdk-testpmd with a single queue, everything seems fine: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i set verbose 5 port stop 0 port config 0 rx_offload vlan_strip on port start 0 start And the packet is correctly displayed: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD But if we use a different number of queues: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 It would be helpful to enable debug logs with EAL argument --log-level='.*',8 so that one can see which 'rx_pkt_burst' method gets selected by the driver. Then one can inspect the implementation of that particular function to see whether
RE: E810 VLAN offload wrong behavior
Hi Ernesto, Good that you've managed to make a difference. However, this is hardly an ideal solution performance-wise, as the plain Rx function does not utilise vector extensions. It is peculiar that the AVX2 version shows this strange behaviour. For the bug report, one may want to use https://bugs.dpdk.org/ -> 'New'. Apart from that, there is a very recent fix [1] that is supposed to be applied on top of the current main branch of 'dpdk-next-net'. May be it is worth trying without the "if 0" Rx function workaround, just to confirm that indeed the issue has something to do with the Rx function and not with device register settings. Thank you. [1] https://patches.dpdk.org/project/dpdk/list/?series=35700 On Tue, 15 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, It worked! ICE_INIT: ice_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port 0. And all the flags are there, with both one and four queues. Any idea on where to report this as a bug in the AVX2 OFFLOAD Vector Rx driver function, as apparently it is? Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 18:12 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, That's interesting. To be honest, I'm not an expert in this driver. So far it does not seem the issue is with the specific Rx function, yet may be it pays to temporarily change "#ifdef RTE_ARCH_X86" to "#if 0" on line [1] in the original source of 24.11.2 and rebuild? Just to make sure the Rx method is a don't care. Thank you. [1] https://github.com/DPDK/dpdk/blob/4c5634de9f933555bab9c64a533d3dca999071cd/d rivers/net/ice/ice_rxtx.c#L3489 On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thank you for your quick answer. I tried to apply the patch but got a compilation error: ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer to incomplete type ?const struct ci_rx_queue? qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, rxq->reg_idx); So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there it compiled. Unfortunately it got consistent, but worse: The selected rx function is the same: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But it now drops the VLAN, without signaling it, in all cases. With 1 queue: ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD I double checked with plain 24.11.2 and indeed with just one queue it is still ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD So I'm confident the lab environment is still good Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 14:49 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thanks for the hint. I ran testpmd with the suggested parameter, but I cannot find where the function gets selected. I attach the full log here. Thanks. The message to look for appears closer to the end of start sequence: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But before investigating the code of that Rx function, perhaps it pays to make sure patch [1] is applied. Everything is so recent that most likely it has not landed 24.11 release yet. Theoretically, the bug the patch is trying to fix has something to do with multi-queue and VLAN stripping. See if you can't apply the patch in 24.11 or rebuild from the current main of dpdk-next-net repository. Thank you. [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html Thanks Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 02:04 To: Ernesto Ruffini Cc: [email protected] Subject: Re: E810 VLAN offload wrong behavior Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: Hi, We found a strange behavior of the E810 VLAN offload. We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and COMMS DDP 1.3.46 The NIC receives an IPv4/UDP packet inside VLAN 300. If we run dpdk-testpmd with a single queue, everything seems fine: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i set verbose 5 port stop 0 port config 0 rx_offload vlan_strip on port start 0 start And the packet is correctly displayed: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_l
RE: E810 VLAN offload wrong behavior
Hi Ivan, It worked! ICE_INIT: ice_set_rx_function(): Rx Burst Bulk Alloc Preconditions are satisfied. Rx Burst Bulk Alloc function will be used on port 0. And all the flags are there, with both one and four queues. Any idea on where to report this as a bug in the AVX2 OFFLOAD Vector Rx driver function, as apparently it is? Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 18:12 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, That's interesting. To be honest, I'm not an expert in this driver. So far it does not seem the issue is with the specific Rx function, yet may be it pays to temporarily change "#ifdef RTE_ARCH_X86" to "#if 0" on line [1] in the original source of 24.11.2 and rebuild? Just to make sure the Rx method is a don't care. Thank you. [1] https://github.com/DPDK/dpdk/blob/4c5634de9f933555bab9c64a533d3dca999071cd/d rivers/net/ice/ice_rxtx.c#L3489 On Mon, 14 Jul 2025, Ernesto Ruffini wrote: > Hi Ivan, > Thank you for your quick answer. > I tried to apply the patch but got a compilation error: > > ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer > to incomplete type ?const struct ci_rx_queue? > qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, > rxq->reg_idx); > > So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there > it compiled. > Unfortunately it got consistent, but worse: > The selected rx function is the same: > ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). > But it now drops the VLAN, without signaling it, in all cases. > With 1 queue: > ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD > RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: > ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD > RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD > > I double checked with plain 24.11.2 and indeed with just one queue it > is still > ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD > RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED > RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD > > So I'm confident the lab environment is still good > > Thank you > Ernesto > > > -Original Message- > From: Ivan Malov > Sent: Monday, July 14, 2025 14:49 > To: Ernesto Ruffini > Cc: [email protected] > Subject: RE: E810 VLAN offload wrong behavior > > Hi Ernesto, > > On Mon, 14 Jul 2025, Ernesto Ruffini wrote: > >> Hi Ivan, >> Thanks for the hint. >> I ran testpmd with the suggested parameter, but I cannot find where >> the function gets selected. >> I attach the full log here. > > Thanks. The message to look for appears closer to the end of start sequence: >> ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). > > But before investigating the code of that Rx function, perhaps it pays > to make sure patch [1] is applied. Everything is so recent that most > likely it has not landed 24.11 release yet. Theoretically, the bug the > patch is trying to fix has something to do with multi-queue and VLAN > stripping. See if you can't apply the patch in 24.11 or rebuild from > the current main of dpdk-next-net repository. > > Thank you. > > [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html > >> Thanks >> Ernesto >> >> -Original Message- >> From: Ivan Malov >> Sent: Monday, July 14, 2025 02:04 >> To: Ernesto Ruffini >> Cc: [email protected] >> Subject: Re: E810 VLAN offload wrong behavior >> >> Hi Ernesto, >> >> On Fri, 11 Jul 2025, Ernesto Ruffini wrote: >> >>> >>> Hi, >>> >>> We found a strange behavior of the E810 VLAN offload. >>> >>> We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 >>> and COMMS DDP 1.3.46 >>> >>> >>> >>> The NIC receives an IPv4/UDP packet inside VLAN 300. >>> >>> If we run dpdk-testpmd with a single queue, everything seems fine: >>> >>> >>> >>> dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i >>> >>> >>> >>> set verbose 5 >>> >>> port stop 0 >>> >>> port config 0 rx_offload vlan_strip on >>> >>> port start 0 >>> >>> start >>> >>> >>> >>> And the packet is correctly displayed: >>> >>> src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - >> type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw
RE: E810 VLAN offload wrong behavior
Hi Ivan, Thanks for the hint. I ran testpmd with the suggested parameter, but I cannot find where the function gets selected. I attach the full log here. Thanks Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 02:04 To: Ernesto Ruffini Cc: [email protected] Subject: Re: E810 VLAN offload wrong behavior Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: > > Hi, > > We found a strange behavior of the E810 VLAN offload. > > We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and > COMMS DDP 1.3.46 > > > > The NIC receives an IPv4/UDP packet inside VLAN 300. > > If we run dpdk-testpmd with a single queue, everything seems fine: > > > > dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i > > > > set verbose 5 > > port stop 0 > > port config 0 rx_offload vlan_strip on > > port start 0 > > start > > > > And the packet is correctly displayed: > > src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: > L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - > Destination UDP port=5398 - Receive queue=0x0 > > ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD > RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED > RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD > > > > But if we use a different number of queues: > > > > dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 It would be helpful to enable debug logs with EAL argument --log-level='.*',8 so that one can see which 'rx_pkt_burst' method gets selected by the driver. Then one can inspect the implementation of that particular function to see whether offloads are handled correctly. Thank you. > > > > The VLAN is in fact removed, but there is no evidence of that: > > > > src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - > type=0x0800 - length=142 - nb_segs=1 - RSS hash=0xfae3080 - RSS > queue=0x0 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: > L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - > Destination UDP port=5398 - Receive queue=0x0 > > ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD > RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD > > > > There was a bug in some previous versions of DPDK about E810 and VLAN offload, but it was fixed. > > Are we doing something wrong or is there a problem with the driver? > > > > Thank you > > Ernesto > > > > > # dpdk-testpmd -a :4b:00.0 -c ff -n 8 --log-level='.*',8 -- -i --rxq=4 --txq=4 EAL: lib.eal log level changed from info to debug EAL: user1 log level changed from info to debug EAL: user2 log level changed from info to debug EAL: user3 log level changed from info to debug EAL: user4 log level changed from info to debug EAL: user5 log level changed from info to debug EAL: user6 log level changed from info to debug EAL: user7 log level changed from info to debug EAL: user8 log level changed from info to debug EAL: lib.node log level changed from info to debug EAL: lib.graph log level changed from info to debug EAL: lib.pipeline log level changed from info to debug EAL: lib.table log level changed from info to debug EAL: lib.pdump log level changed from notice to debug EAL: lib.port log level changed from info to debug EAL: lib.fib log level changed from info to debug EAL: lib.vhost.fdset log level changed from info to debug EAL: lib.vhost.config log level changed from info to debug EAL: lib.vhost.data log level changed from warning to debug EAL: lib.vhost.crypto log level changed from info to debug EAL: lib.stack log level changed from notice to debug EAL: lib.sched log level changed from info to debug EAL: lib.reorder log level changed from info to debug EAL: lib.rib log level changed from info to debug EAL: lib.mldev log level changed from info to debug EAL: lib.regexdev log level changed from info to debug EAL: lib.rawdev log level changed from info to debug EAL: lib.power log level changed from info to debug EAL: lib.lpm log level changed from info to debug EAL: lib.latencystats log level changed from info to debug EAL: lib.ip_frag log level changed from info to debug EAL: lib.gpudev log level changed from notice to debug EAL: lib.eventdev.adapter.timer log level changed from notice to debug EAL: lib.eventdev.adapter.timer.svc log level changed from notice to debug EAL: lib.eventdev log level changed from info to debug EAL: lib.efd log level changed from info to debug EAL: lib.dmadev log level changed from info to debug EAL: lib.cryptodev log level changed from info to debu
RE: E810 VLAN offload wrong behavior
Hi Ernesto, That's interesting. To be honest, I'm not an expert in this driver. So far it does not seem the issue is with the specific Rx function, yet may be it pays to temporarily change "#ifdef RTE_ARCH_X86" to "#if 0" on line [1] in the original source of 24.11.2 and rebuild? Just to make sure the Rx method is a don't care. Thank you. [1] https://github.com/DPDK/dpdk/blob/4c5634de9f933555bab9c64a533d3dca999071cd/drivers/net/ice/ice_rxtx.c#L3489 On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thank you for your quick answer. I tried to apply the patch but got a compilation error: ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer to incomplete type ‘const struct ci_rx_queue’ qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, rxq->reg_idx); So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there it compiled. Unfortunately it got consistent, but worse: The selected rx function is the same: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But it now drops the VLAN, without signaling it, in all cases. With 1 queue: ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD I double checked with plain 24.11.2 and indeed with just one queue it is still ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD So I'm confident the lab environment is still good Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 14:49 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thanks for the hint. I ran testpmd with the suggested parameter, but I cannot find where the function gets selected. I attach the full log here. Thanks. The message to look for appears closer to the end of start sequence: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But before investigating the code of that Rx function, perhaps it pays to make sure patch [1] is applied. Everything is so recent that most likely it has not landed 24.11 release yet. Theoretically, the bug the patch is trying to fix has something to do with multi-queue and VLAN stripping. See if you can't apply the patch in 24.11 or rebuild from the current main of dpdk-next-net repository. Thank you. [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html Thanks Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 02:04 To: Ernesto Ruffini Cc: [email protected] Subject: Re: E810 VLAN offload wrong behavior Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: Hi, We found a strange behavior of the E810 VLAN offload. We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and COMMS DDP 1.3.46 The NIC receives an IPv4/UDP packet inside VLAN 300. If we run dpdk-testpmd with a single queue, everything seems fine: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i set verbose 5 port stop 0 port config 0 rx_offload vlan_strip on port start 0 start And the packet is correctly displayed: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD But if we use a different number of queues: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 It would be helpful to enable debug logs with EAL argument --log-level='.*',8 so that one can see which 'rx_pkt_burst' method gets selected by the driver. Then one can inspect the implementation of that particular function to see whether offloads are handled correctly. Thank you. The VLAN is in fact removed, but there is no evidence of that: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - RSS hash=0xfae3080 - RSS queue=0x0 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD There was a bug in some previous versions of DPDK about E810 and VLAN offload, but it was fixed. Are we doing something wrong or is there a problem with the driver? Thank you Ernesto
RE: E810 VLAN offload wrong behavior
Hi Ivan, Thank you for your quick answer. I tried to apply the patch but got a compilation error: ../drivers/net/ice/ice_ethdev.c:5003:71: error: dereferencing pointer to incomplete type const struct ci_rx_queue qrx_context_offset = QRX_CONTEXT(ICE_L2TSEL_QRX_CONTEXT_REG_IDX, rxq->reg_idx); So I did "git clone git://dpdk.org/next/dpdk-next-net" and from there it compiled. Unfortunately it got consistent, but worse: The selected rx function is the same: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But it now drops the VLAN, without signaling it, in all cases. With 1 queue: ol_flags: RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD With 4 queues: ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD I double checked with plain 24.11.2 and indeed with just one queue it is still ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD So I'm confident the lab environment is still good Thank you Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 14:49 To: Ernesto Ruffini Cc: [email protected] Subject: RE: E810 VLAN offload wrong behavior Hi Ernesto, On Mon, 14 Jul 2025, Ernesto Ruffini wrote: > Hi Ivan, > Thanks for the hint. > I ran testpmd with the suggested parameter, but I cannot find where > the function gets selected. > I attach the full log here. Thanks. The message to look for appears closer to the end of start sequence: > ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But before investigating the code of that Rx function, perhaps it pays to make sure patch [1] is applied. Everything is so recent that most likely it has not landed 24.11 release yet. Theoretically, the bug the patch is trying to fix has something to do with multi-queue and VLAN stripping. See if you can't apply the patch in 24.11 or rebuild from the current main of dpdk-next-net repository. Thank you. [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html > Thanks > Ernesto > > -Original Message- > From: Ivan Malov > Sent: Monday, July 14, 2025 02:04 > To: Ernesto Ruffini > Cc: [email protected] > Subject: Re: E810 VLAN offload wrong behavior > > Hi Ernesto, > > On Fri, 11 Jul 2025, Ernesto Ruffini wrote: > >> >> Hi, >> >> We found a strange behavior of the E810 VLAN offload. >> >> We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and >> COMMS DDP 1.3.46 >> >> >> >> The NIC receives an IPv4/UDP packet inside VLAN 300. >> >> If we run dpdk-testpmd with a single queue, everything seems fine: >> >> >> >> dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i >> >> >> >> set verbose 5 >> >> port stop 0 >> >> port config 0 rx_offload vlan_strip on >> >> port start 0 >> >> start >> >> >> >> And the packet is correctly displayed: >> >> src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - > type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: > L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: >> L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - >> Destination UDP port=5398 - Receive queue=0x0 >> >> ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD >> RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED >> RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD >> >> >> >> But if we use a different number of queues: >> >> >> >> dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 > > It would be helpful to enable debug logs with EAL argument > --log-level='.*',8 so that one can see which 'rx_pkt_burst' method > gets selected by the driver. > Then one can inspect the implementation of that particular function to > see whether offloads are handled correctly. > > Thank you. > >> >> >> >> The VLAN is in fact removed, but there is no evidence of that: >> >> >> >> src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - >> type=0x0800 - length=142 - nb_segs=1 - RSS hash=0xfae3080 - RSS >> queue=0x0 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: >> L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - >> Destination UDP port=5398 - Receive queue=0x0 >> >> ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD >> RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD >> >> >> >> There was a bug in some previous versions of DPDK about E810 and VLAN > offload, but it was fixed. >> >> Are we doing something wrong or is there a problem with the driver? >> >> >> >> Thank you >> >> Ernesto >> >> >> >> >> >
RE: E810 VLAN offload wrong behavior
Hi Ernesto, On Mon, 14 Jul 2025, Ernesto Ruffini wrote: Hi Ivan, Thanks for the hint. I ran testpmd with the suggested parameter, but I cannot find where the function gets selected. I attach the full log here. Thanks. The message to look for appears closer to the end of start sequence: ICE_DRIVER: ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 0). But before investigating the code of that Rx function, perhaps it pays to make sure patch [1] is applied. Everything is so recent that most likely it has not landed 24.11 release yet. Theoretically, the bug the patch is trying to fix has something to do with multi-queue and VLAN stripping. See if you can't apply the patch in 24.11 or rebuild from the current main of dpdk-next-net repository. Thank you. [1] https://mails.dpdk.org/archives/dev/2025-June/320530.html Thanks Ernesto -Original Message- From: Ivan Malov Sent: Monday, July 14, 2025 02:04 To: Ernesto Ruffini Cc: [email protected] Subject: Re: E810 VLAN offload wrong behavior Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: Hi, We found a strange behavior of the E810 VLAN offload. We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and COMMS DDP 1.3.46 The NIC receives an IPv4/UDP packet inside VLAN 300. If we run dpdk-testpmd with a single queue, everything seems fine: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i set verbose 5 port stop 0 port config 0 rx_offload vlan_strip on port start 0 start And the packet is correctly displayed: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD But if we use a different number of queues: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 It would be helpful to enable debug logs with EAL argument --log-level='.*',8 so that one can see which 'rx_pkt_burst' method gets selected by the driver. Then one can inspect the implementation of that particular function to see whether offloads are handled correctly. Thank you. The VLAN is in fact removed, but there is no evidence of that: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - RSS hash=0xfae3080 - RSS queue=0x0 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD There was a bug in some previous versions of DPDK about E810 and VLAN offload, but it was fixed. Are we doing something wrong or is there a problem with the driver? Thank you Ernesto
Re: E810 VLAN offload wrong behavior
Hi Ernesto, On Fri, 11 Jul 2025, Ernesto Ruffini wrote: Hi, We found a strange behavior of the E810 VLAN offload. We are running DPDK 24.11.2 with ice driver 1.15.4, firmware 4.60 and COMMS DDP 1.3.46 The NIC receives an IPv4/UDP packet inside VLAN 300. If we run dpdk-testpmd with a single queue, everything seems fine: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i set verbose 5 port stop 0 port config 0 rx_offload vlan_strip on port start 0 start And the packet is correctly displayed: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - VLAN tci=0x12c - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_VLAN RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_VLAN_STRIPPED RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD But if we use a different number of queues: dpdk-testpmd -a :4b:00.0 -c ff -n 8 -- -i --rxq=4 --txq=4 It would be helpful to enable debug logs with EAL argument --log-level='.*',8 so that one can see which 'rx_pkt_burst' method gets selected by the driver. Then one can inspect the implementation of that particular function to see whether offloads are handled correctly. Thank you. The VLAN is in fact removed, but there is no evidence of that: src=08:02:03:04:05:06 - dst=00:1A:CA:01:00:96 - pool=mb_pool_0 - type=0x0800 - length=142 - nb_segs=1 - RSS hash=0xfae3080 - RSS queue=0x0 - hw ptype: L2_ETHER L3_IPV4_EXT_UNKNOWN L4_UDP - sw ptype: L2_ETHER L3_IPV4 L4_UDP - l2_len=14 - l3_len=20 - l4_len=8 - Destination UDP port=5398 - Receive queue=0x0 ol_flags: RTE_MBUF_F_RX_RSS_HASH RTE_MBUF_F_RX_L4_CKSUM_GOOD RTE_MBUF_F_RX_IP_CKSUM_GOOD RTE_MBUF_F_RX_OUTER_L4_CKSUM_GOOD There was a bug in some previous versions of DPDK about E810 and VLAN offload, but it was fixed. Are we doing something wrong or is there a problem with the driver? Thank you Ernesto
