Hey everybody, My application relies on some control plane apps that need to talk to the network through ports that are controlled by DPDK. Historically, we've used KNI for that, which lets the control plane applications use the normal Linux network stack and interact with it with common tools. Recently, I've been looking into leaving KNI and moving to virtio for the same purpose. However, I find that I can't move traffic through it as quickly as I could with KNI.
Specifically, the relevant portions of my dataplane application move data around roughly like this: NIC<->PMD<->packet ingestor (recognizes control plane traffic) <-> KNI/virtio thread <-> KNI/virtio What I find is that if my control plane application is mostly sending traffic out to the network, I can achieve roughly the same bandwidth through virtio that I did through KNI. If I'm primarily reading from the network, however, I only get about half of the bandwidth I used to get from KNI. Also, I see congestion when enqueueing for virtio. The congestion isn't alleviated by increasing queue sizes, so I presume I'm not dealing with occasional bursts, but general slowness. Obviously, optimization is a complex subject, and I'm only describing a small corner of a more complex codebase. But, for those of you who have more experience with virtio under DPDK than I do, if you were presented with a problem like this, what is the handful of questions you ask, or big wins you'd seek out first? A few details about my application: - Physical hardware, with isolated CPU cores for the dataplane, not a VM - We're using the virtio library that comes with Ubuntu, not a custom/DPDK one - Checksum generation and validation are enabled at the NIC, but TSO and LRO are not, as they don't play nicely with the rest of our application Thanks in advance, - Matt
