On 30/07/2024 20:00, Panuski, Patrick via USRP-users wrote:
Hi USRP users,
I’m doing a timed transmission and I’m tracking how many samples are
buffered before the transmission starts. I do this by calling send()
with tx_metadata timestamps in the future and accumulating the return
value from send() until it starts returning 0. Send() returns 0 until
the start time occurs, after which the send() call starts returning a
non-zero number again indicating samples are successfully being sent
from the host. Based on this accumulated number, the total number of
buffered samples is only 63,488 (=248 kB of data) despite having the
linux kernel network buffer sizes (net.core.wmem_max and wmem_default
settings) set to 50 MB. Is this expected? How can I increase the
number of buffered samples between the host and device? Latency is not
a concern. Should I be adjusting the samples per buffer or using
timeout=0 here?
Background:
I’m using an X440 and my end goal is to get at least 4 channels
coherently transmitting (i.e. stable phase relationship for an entire
burst/session) at 50 MSps or greater on each channel for runs on the
order of minutes. The problem I’m running into is that during
transmission at these sample rates, it is very likely that one of the
channels underflows, and even if it’s just a single underflow, it
breaks the phase relationship with the other channels for the rest of
the run since I’m using multithreaded streamers in a custom C++
program. The real core of my problem is trying to figure out why I’m
underflowing at all. My hardware setup seems more than capable (see
below); CPU usage per active core stays below 50% or so, the network
traffic doesn’t and shouldn’t come anywhere close to the 4x 10GbE
capacity, and the file reads from SSD are staying far ahead of the
sender. Based on my testing, I’m virtually certain that a large-ish
buffer between my UHD application and the X440 would solve all
underflow issues, but it seems right now the buffer is only 248 kB, or
about 1 ms @ 50 MSps. I have occasionally gotten a full transmission
to complete without a single underflow. The onset of the underflow
seems to randomly happen, which is another indicator that a larger
buffer will help smooth these inconsistencies.
Setup info:
* UHD 4.6 on host and device, FPGA is running a customized image
(X4_400 stock image but with the RAM replay block replaced with
DDC/DUC)
* Host Hardware – AMD threadripper, cores set to performance, Intel
E810-CQDA2 NIC configured to split 1x 100GbE port into 4x logical
10GbE ports
* Host Software – Ubuntu 20.04, running a C++ program that spawns 2
threads for each transmit channel; 1 “producer” for reading a file
into a series of very large buffers in memory, and the other
“consumer” for moving a pointer to the correct point in those
buffers and calling send(). The producer threads start early and I
have proven that they never fall behind the consumers. I’ve
increased the net.core.wmem_max and wmem_default values to 50 MB,
enabled tx pause frames on the NIC, maxed out the tx/rx
descriptors on the NIC, and followed all the other tuning tips
and tricks. I am not running DPDK as I didn’t think it would be
necessary at these sample rates, although I could be wrong about
that.
Any help here would be much appreciated, thanks!
Patrick
_______________________________________________
USRP-users mailing list --usrp-users@lists.ettus.com
To unsubscribe send an email tousrp-users-le...@lists.ettus.com
You can look at /proc/net/udp when you have things running to check
queue sizes, etc.
Keep in mind that simply changing /etc/sysctl.conf will not do
*anything* until the next reboot.
_______________________________________________
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-le...@lists.ettus.com
