Re: [PATCH 0/3] iothread and irqfd support
I also came across the Interrupt Coalescing feature in NVMe (5.21.1.8 in Spec 1.4c). It seems to suggest an algorithm based on thresholds for the maximum interrupt delay time and minimum number of coalesced interrupts. Klaus, based on your experience, how does this algorithm work? Is is commonly used by controllers and drivers?
Re: [PATCH 0/3] iothread and irqfd support
virtio-blk's dataplane BH completion batching mechanism is not enabled by default and the performance results are mixed. If you develop a different mechanism from scratch I think there's a good chance it would work better :). This looks like a queuing theory problem to me. It should be possible to model IOPS as a function of some parameters and then hopefully find simple rules to optimize IOPS by adjusting some of the parameters at runtime. I haven't looked into this much myself though, so I don't have any concrete suggestion. The basic idea is that as long as events occur at a minimum rate then they can be batched to maximize throughput without sacrificing too much latency. If the rate drops then the device cannot hold back events. Another place to look for inspiration is network cards. In Linux it's common to use the NAPI framework to disable further receive interrupts and then poll until the receive queue becomes empty. Transmit completions can also be mitigated, but I'm not sure what the most common approach is there. Stefan
Re: [PATCH 0/3] iothread and irqfd support
On 10/12/2022 10:39 PM, Klaus Jensen wrote: I have been meaning to pick it up, but I got side-tracked. The polling performance drop needs to be address as we discussed offline. But the v4 looks pretty good and I can pick that up without the polling support for now. I've been using the v4 without polling for my daily work. It worked pretty well for my test workloads. I'm not sure what needs to be done for the polling problem. I can try to add a completion batching mechanism with BH, similar to virtio-blk. Do you think this is the right direction?
Re: [PATCH 0/3] iothread and irqfd support
On Okt 12 10:28, Stefan Hajnoczi wrote: > On Fri, 26 Aug 2022 at 07:18, Jinhao Fan wrote: > > > > This patch series adds support for using a seperate iothread for NVMe > > IO emulation, which brings the potential of applying polling. The > > first two patches implements support for irqfd, which solves thread > > safety problems for interrupt emulation outside the main loop thread. > > > > Jinhao Fan (3): > > hw/nvme: support irq(de)assertion with eventfd > > hw/nvme: use KVM irqfd when available > > hw/nvme: add iothread support > > Hi, > What is the status of this series? > I have been meaning to pick it up, but I got side-tracked. The polling performance drop needs to be address as we discussed offline. But the v4 looks pretty good and I can pick that up without the polling support for now. signature.asc Description: PGP signature
Re: [PATCH 0/3] iothread and irqfd support
On Fri, 26 Aug 2022 at 07:18, Jinhao Fan wrote: > > This patch series adds support for using a seperate iothread for NVMe > IO emulation, which brings the potential of applying polling. The > first two patches implements support for irqfd, which solves thread > safety problems for interrupt emulation outside the main loop thread. > > Jinhao Fan (3): > hw/nvme: support irq(de)assertion with eventfd > hw/nvme: use KVM irqfd when available > hw/nvme: add iothread support Hi, What is the status of this series? Stefan
[PATCH 0/3] iothread and irqfd support
This patch series adds support for using a seperate iothread for NVMe IO emulation, which brings the potential of applying polling. The first two patches implements support for irqfd, which solves thread safety problems for interrupt emulation outside the main loop thread. Jinhao Fan (3): hw/nvme: support irq(de)assertion with eventfd hw/nvme: use KVM irqfd when available hw/nvme: add iothread support hw/nvme/ctrl.c | 335 +++ hw/nvme/ns.c | 21 ++- hw/nvme/nvme.h | 12 +- hw/nvme/trace-events | 3 + 4 files changed, 342 insertions(+), 29 deletions(-) -- 2.25.1
