On Mon, Jul 21, 2025 at 05:03:07PM +0800, Jason Xing wrote: > On Mon, Jul 21, 2025 at 3:25 PM Larysa Zaremba <[email protected]> > wrote: > > > > On Sat, Jul 19, 2025 at 01:26:18PM +0800, Jason Xing wrote: > > > On Fri, Jul 18, 2025 at 6:27 PM Larysa Zaremba <[email protected]> > > > wrote: > > > > > > > > On Wed, Jul 16, 2025 at 11:41:42AM +0800, Jason Xing wrote: > > > > > Hi all, > > > > > > > > > > I'm currently faced with one tough issue caused by zero copy mode in > > > > > xsk with ixgbe driver loaded. The case is that if we use xdpsock to > > > > > send descs, nearly at the same time normal packets from other tx > > > > > queues cannot be transmitted/completed at all. > > > > > > > > > > Here is how I try: > > > > > 1. run iperf or ping to see if the transmission is successful. > > > > > 2. then run "timeout 5 ./xdpsock -i enp2s0f0 -t -z -s 64" > > > > > > > > > > You will obviously find the whole machine loses connection. It can > > > > > only recover as soon as the xdpsock is stopped due to timeout. > > > > > > > > > > I tried a lot and then traced down to this line in ixgbe driver: > > > > > ixgbe_clean_tx_irq() > > > > > -> if (!(eop_desc->wb.status & cpu_to_le32(IXGBE_TXD_STAT_DD))) > > > > > break; > > > > > The above line always 'breaks' the sending process. > > > > > > > > > > I also managed to make the external ixgbe 6.15 work and it turned out > > > > > to be the same issue as before. > > > > > > > > > > I have no idea on how to analyze further in this driver. Could someone > > > > > point out a direction that I can take? Is it a known issue? > > > > > > > > > > Thanks, > > > > > Jason > > > > > > > > > > > > > I was able to reproduce the described behaviour, xdpsock does break the > > > > IP > > > > communication. However, in my case this was not because of ixgbe not > > > > being able > > > > to send, but because of queue 0 RX packets being dropped, which is the > > > > indended > > > > outcome in xdpsock, even in Tx only mode. > > > > > > Thanks for your feedback. It would be great if you could elaborate > > > more on this. How did you spot that it's queue 0 that causes the > > > problem? > > > > If you do not specify -q parameter, xdpsock loads on the queue pair 0. > > > > > Why is xdpsock breaking IP communication intended? > > > > Because when a packet arrives on the AF_XDP-managed queue (0 in this case), > > the > > default xdpsock XDP program provided by libxdp returns XDP_REDIRECT even in > > tx-only mode, XDP_PASS for all other queues (1-39). XDP_REDIRECT results in > > a > > packet leaving the kernel network stack, it is now managed by the AF_XDP > > userspace program. I think it is possible to modify libxdp to return > > XDP_PASS > > when the socket is tx-only. > > > > > > > > When you try i40e, you will find the connection behaves normally. Ping > > > can work as usual. As I depicted before, with ixgbe driver, ping even > > > doesn't work at all. > > > > I think this is due to RSS configuration, ping packets on i40e go to another > > queue. > > Thanks so much for your detailed explanation. > > But, I still doubt it's not the reason why xdpsock breaks with ixgbe > driver loaded because I tried the following commands: > 1. ping <ip> > 2. timeout 3 ./xdpsock -i enp2s0f0np0 -t -q 11 -z -s 64 > Whatever the queue value I adjusted, ping always fails with > "Destination Host Unreachable" warning.
First I would make sure, if the problem is from Rx or Tx. This is hard to do with IP-level applications, because they fail if any of those does not work. Please, try sending MAC packet with scapy using a following command: sendp(Ether(src="<src MAC>", dst="<dst MAC>")/IP(src="<any IP>", dst="<another IP>")/UDP(sport=2000, dport=9091)/Raw(load="xdp"), iface="<src ifname>") First, send this from host to link partner while running `tcpdump -nn -e -p -i <ifname>` on link partner. Then send this from LK to host while running tcpdump on host. In both cases, note whether the packet shows up in tcpdump with and without xdpsock loaded on the host. > > Thanks, > Jason
