On Mon, Jul 21, 2025 at 5:25 PM Larysa Zaremba <[email protected]> wrote: > > On Mon, Jul 21, 2025 at 05:03:07PM +0800, Jason Xing wrote: > > On Mon, Jul 21, 2025 at 3:25 PM Larysa Zaremba <[email protected]> > > wrote: > > > > > > On Sat, Jul 19, 2025 at 01:26:18PM +0800, Jason Xing wrote: > > > > On Fri, Jul 18, 2025 at 6:27 PM Larysa Zaremba > > > > <[email protected]> wrote: > > > > > > > > > > On Wed, Jul 16, 2025 at 11:41:42AM +0800, Jason Xing wrote: > > > > > > Hi all, > > > > > > > > > > > > I'm currently faced with one tough issue caused by zero copy mode in > > > > > > xsk with ixgbe driver loaded. The case is that if we use xdpsock to > > > > > > send descs, nearly at the same time normal packets from other tx > > > > > > queues cannot be transmitted/completed at all. > > > > > > > > > > > > Here is how I try: > > > > > > 1. run iperf or ping to see if the transmission is successful. > > > > > > 2. then run "timeout 5 ./xdpsock -i enp2s0f0 -t -z -s 64" > > > > > > > > > > > > You will obviously find the whole machine loses connection. It can > > > > > > only recover as soon as the xdpsock is stopped due to timeout. > > > > > > > > > > > > I tried a lot and then traced down to this line in ixgbe driver: > > > > > > ixgbe_clean_tx_irq() > > > > > > -> if (!(eop_desc->wb.status & cpu_to_le32(IXGBE_TXD_STAT_DD))) > > > > > > break; > > > > > > The above line always 'breaks' the sending process. > > > > > > > > > > > > I also managed to make the external ixgbe 6.15 work and it turned > > > > > > out > > > > > > to be the same issue as before. > > > > > > > > > > > > I have no idea on how to analyze further in this driver. Could > > > > > > someone > > > > > > point out a direction that I can take? Is it a known issue? > > > > > > > > > > > > Thanks, > > > > > > Jason > > > > > > > > > > > > > > > > I was able to reproduce the described behaviour, xdpsock does break > > > > > the IP > > > > > communication. However, in my case this was not because of ixgbe not > > > > > being able > > > > > to send, but because of queue 0 RX packets being dropped, which is > > > > > the indended > > > > > outcome in xdpsock, even in Tx only mode. > > > > > > > > Thanks for your feedback. It would be great if you could elaborate > > > > more on this. How did you spot that it's queue 0 that causes the > > > > problem? > > > > > > If you do not specify -q parameter, xdpsock loads on the queue pair 0. > > > > > > > Why is xdpsock breaking IP communication intended? > > > > > > Because when a packet arrives on the AF_XDP-managed queue (0 in this > > > case), the > > > default xdpsock XDP program provided by libxdp returns XDP_REDIRECT even > > > in > > > tx-only mode, XDP_PASS for all other queues (1-39). XDP_REDIRECT results > > > in a > > > packet leaving the kernel network stack, it is now managed by the AF_XDP > > > userspace program. I think it is possible to modify libxdp to return > > > XDP_PASS > > > when the socket is tx-only. > > > > > > > > > > > When you try i40e, you will find the connection behaves normally. Ping > > > > can work as usual. As I depicted before, with ixgbe driver, ping even > > > > doesn't work at all. > > > > > > I think this is due to RSS configuration, ping packets on i40e go to > > > another > > > queue. > > > > Thanks so much for your detailed explanation. > > > > But, I still doubt it's not the reason why xdpsock breaks with ixgbe > > driver loaded because I tried the following commands: > > 1. ping <ip> > > 2. timeout 3 ./xdpsock -i enp2s0f0np0 -t -q 11 -z -s 64 > > Whatever the queue value I adjusted, ping always fails with > > "Destination Host Unreachable" warning. > > First I would make sure, if the problem is from Rx or Tx. This is hard to do
I ran tcpdump xxx & on the other machine (receiver) to see if there were any incoming icmp packets first. It turned out to be no incoming data at all. So I presume the problem happens on the tx side. > with IP-level applications, because they fail if any of those does not work. > > Please, try sending MAC packet with scapy using a following command: > sendp(Ether(src="<src MAC>", dst="<dst MAC>")/IP(src="<any IP>", > dst="<another IP>")/UDP(sport=2000, dport=9091)/Raw(load="xdp"), iface="<src > ifname>") > > First, send this from host to link partner while running `tcpdump -nn -e -p -i > <ifname>` on link partner. > > Then send this from LK to host while running tcpdump on host. > > In both cases, note whether the packet shows up in tcpdump with and without > xdpsock loaded on the host. Thanks for your guidance. But it probably takes me more time than expected due to the whole connection being totally broken while xdpsock is running :( I ran tcpdump xxx > log & while running xdpsock. Later, I found that there are as many packets as w/o xdpscok. Thanks, Jason
