> From: fwefew 4t4tg <[email protected]> > Sent: Monday, March 6, 2023 1:01 AM > To: [email protected] > Subject: Using rdtsc to timestamp RTT of packets
> I convinced myself that a viable way to measure timestamps between a request > packet and its response packet can be the difference between two Intel rdtsc > calls > The restrictions to valid use include: • RTT (time difference) must be calculated on the same CORE For all CPU generations that you likely care-about (check lscpu | egrep "constant_tsc|nonstop_tsc" output) all CPUs on the same socket have a single TSC, and never stop. > • DPDK gives me the frequency rte_rdtsc_cycles(); this way I can convert from > a rdtsc difference to elapsed time Correct, this gives a reference to convert "N TSC ticks" to real-world time "X of a second". <snip> > • The TSC is not always invariant TSC is invariant/constant and non-stop based on the lscpu check above - so you can confirm your platform. > • And of course context switches (if a thread is not pinned to a core) will > invalidate any time difference Context switches will not invalidate your timestamp - but if you're reacting to an interrupt which *then* timestamps "end" of measurement, then yes it is invalid/noisy. I'm trying to say, the context-switch jitter will make the measurement much more noisy - but "invalid" depends on requirements. Typically as DPDK has pinned & polling cores, this isn't a problem. > Now, of course, Mellanox can report time stamps. Is it actually possible to > get HW NIC timestamps reported for every packet sent and received without > overburdening the NIC? > Based on what I can see for my case (Connect 4 LX) > resolution is nanoseconds. So I am tempted to not fool around with rdtsc and > just use NIC timestamps. What is praxis in DPDK programming when one needs RTTs? Some NICs have 1588 capabilities; these can timestamp *specific* packets on the wire to NS resolution. They typically can not timestamp *every* packet, or even 1 every ~million packets... but if measuring every Nth packet (with large N) then that might be worth trying, if the resolution and "closest-to-the-wire" timestamp are really required. I would recommend going with a SW + TSC approach, for flexibility & ease of use; deploying in a cloud/virtio NIC environment will mean that ptp1588 or HW features are not available, so TSC is the "only" way to go then. PTP 1588: https://doc.dpdk.org/api/rte__ethdev_8h.html#ad7aaa6d846a6e71edb4f6c737cdf60c7 and examples/ptpclient, and a HW PTP timestamping paper by "moongen" authors for reference (section 6) https://arxiv.org/pdf/1410.3322.pdf On SW traffic-generation, latency, histograms etc, I've presented at DPDK Userspace last September on the topic: https://www.youtube.com/watch?v=Djcjq59H1uo My conclusion (in the Q&A at end) is that for my use-cases, using TSC was a simpler/faster/easier option, and that resolution was much higher (and more than I required) for RTT in packet processing applications. Hope that helps! -Harry PS: apologies for formatting, it seems some HTML like mail was sent originally? Please send plain-text emails to mailing-lists.
