On Wed, Aug 5, 2026 at 10:40 PM Liu, Changcheng <[email protected]> wrote: > > On Wed, Aug 05, 2026 at 01:45:58PM -0400, Stefan Hajnoczi wrote: > > > > On Sat, Aug 1, 2026 at 2:21 AM Liu, Changcheng <[email protected]> wrote: > > > > > > When creating many virtio-blk devices, probe starts failing with > > > -ENOSPC (-28) because the system runs out of interrupt vectors: > > > > > > virtio_blk virtioNNN: probe with driver virtio_blk failed with error -28 > > > > > > By default virtio-blk uses managed IRQ affinity, which reserves an > > > interrupt vector on every CPU for each device (about nr_cpus vectors per > > > device). On a host with many CPUs and many devices this exhausts the > > > vectors long before all devices are probed. > > > > > > Add use_irq_affinity (default true, no behaviour change). Set it to 0 to > > > use unmanaged interrupts, so each device only uses a couple of vectors > > > instead of one per CPU, allowing far more devices to probe. > > > > > > Signed-off-by: Liu, Changcheng <[email protected]> > > > > There is already a num_request_queues module parameter for cases where > > the user wishes to reduce the number of virtqueues. Did you benchmark > > that and decide the performance of many queues sharing a single irq > > makes it worth adding another module parameter? > > > > Stefan > > num_request_queues does not address this case because each virtio-blk device > already has only one request virtqueue when the failure occurs, so the queue > count cannot be reduced further.
Okay. > The issue is reproduced after probing approximately 800 virtio-blk devices on > a 64-core bare-metal host. With managed IRQ affinity, the IRQ-vector > reservations > are eventually exhausted and probing additional devices fails with -ENOSPC. > Disabling managed IRQ affinity avoids these per-CPU vector reservations and > allows more devices to be probed successfully. I don't follow. My understanding was that non-managed IRQ vectors are allocated by request_irq(), which is called during virtio_find_vqs(). Probing 800 devices would still require 800 * (1 virtqueue irq + 1 config change irq) = 1,600 vectors. How come non-managed IRQs do not hit the limit here? Thanks, Stefan > This patch does not cause multiple virtqueues to share a single IRQ. Passing a > NULL affinity descriptor only disables managed IRQ affinity. Regardless of the > number of virtqueues, virtio-pci still first attempts to allocate a separate > interrupt vector for each interrupt-driven virtqueue. Only if per-VQ vector > allocation fails does the existing virtio-pci fallback share a vector among > the virtqueues, and that fallback is independent of this patch. > > The new parameter defaults to enabled, so the existing driver behavior remains > unchanged unless the option is explicitly disabled. The option is intended for > environments with a very large number of devices, particularly for functional > device testing. > > - Changcheng > > > > > > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c > > > index 32bf3ba07a9d..184f1c1f4485 100644 > > > --- a/drivers/block/virtio_blk.c > > > +++ b/drivers/block/virtio_blk.c > > > @@ -41,6 +41,11 @@ static unsigned int poll_queues; > > > module_param(poll_queues, uint, 0644); > > > MODULE_PARM_DESC(poll_queues, "The number of dedicated virtqueues for > > > polling I/O"); > > > > > > +static bool use_irq_affinity = true; > > > +module_param(use_irq_affinity, bool, 0644); > > > +MODULE_PARM_DESC(use_irq_affinity, > > > + "Use managed IRQ affinity for virtqueues (default: > > > true)"); > > > + > > > static int major; > > > static DEFINE_IDA(vd_index_ida); > > > > > > @@ -1016,7 +1021,8 @@ static int init_vq(struct virtio_blk *vblk) > > > } > > > > > > /* Discover virtqueues and write information to configuration. */ > > > - err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, &desc); > > > + err = virtio_find_vqs(vdev, num_vqs, vqs, vqs_info, > > > + use_irq_affinity ? &desc : NULL); > > > if (err) > > > goto out; > > > > > > -- > > > 2.43.7 > > > > > >
