This series represents a major enabler for two AioContext projects: true multiqueue (i.e. multiple AioContexts for the same BDS) and thread-safe virtio-scsi dataplane.
The patches decouple AioContext's internal locking for the BDS's locking in two steps. Patches 1 to 9 make aio_poll and the iothread run without taking any external locks---only user callbacks are protected by aio_context_acquire/aio_context_release. Patches 12 to 16 then push the lock down to the callbacks themselves, so that the AioContext lock becomes nothing more than a convenience. For multiqueue the next step would be to have fine-grained locks so that e.g. each dataplane thread can proceed independently. For thread-safe virtio-scsi dataplane, most of dma-helpers.c is already running without taking the AioContext lock, and all that's left is to ensure that dma-helpers.c code is _always_ run without that lock taken. Of course the latter is a much smaller endeavor. I'm fairly confident that patches 1 to 11 are correct and would like to get them in before patches 12 to 18. (This second half is much less tested, especially I have not tested Win32 nor quorum). I'm nevertheless posting the whole thing as RFC and not just for review, since I want to discuss this and the next steps forward at KVM Forum. Applies on top of the three SCSI patches I have just sent. Paolo Paolo Bonzini (18): iothread: release iothread around aio_poll aio: rename bh_lock to list_lock qemu-thread: introduce QemuLockCnt aio: make ctx->list_lock a QemuLockCnt, subsuming ctx->walking_bh aio: tweak walking in dispatch phase aio-posix: remove walking_handlers, protecting AioHandler list with list_lock aio-win32: remove walking_handlers, protecting AioHandler list with list_lock aio: document locking aio: push aio_context_acquire/release down to dispatching async: optimize aio_bh_poll qemu-timer: optimize timerlist_run_timers block: explicitly acquire aiocontext in callbacks that need it block: explicitly acquire aiocontext in bottom halves that need it block: explicitly acquire aiocontext in timers that need it quorum: use atomics for rewrite_count quorum: split quorum_fifo_aio_cb from quorum_aio_cb block: explicitly acquire aiocontext in aio callbacks that need it aio: update locking documentation aio-posix.c | 78 +++++----- aio-win32.c | 97 +++++++------ async.c | 63 ++++---- block/blkverify.c | 6 +- block/curl.c | 43 ++++-- block/gluster.c | 2 + block/io.c | 7 + block/iscsi.c | 10 ++ block/linux-aio.c | 14 +- block/mirror.c | 12 +- block/nbd-client.c | 14 +- block/nfs.c | 10 ++ block/qed.c | 2 + block/quorum.c | 60 +++++--- block/sheepdog.c | 29 ++-- block/ssh.c | 45 +++--- block/throttle-groups.c | 2 + block/win32-aio.c | 8 +- dma-helpers.c | 7 +- docs/lockcnt.txt | 308 ++++++++++++++++++++++++++++++++++++++++ docs/multiple-iothreads.txt | 95 ++++++++++--- hw/block/dataplane/virtio-blk.c | 2 + hw/block/virtio-blk.c | 8 ++ hw/scsi/scsi-bus.c | 2 + hw/scsi/scsi-disk.c | 18 +++ hw/scsi/scsi-generic.c | 20 ++- hw/scsi/virtio-scsi-dataplane.c | 6 + include/block/aio.h | 33 ++--- include/qemu/thread.h | 17 +++ iothread.c | 11 +- nbd.c | 4 + qemu-coroutine-sleep.c | 5 + qemu-timer.c | 4 + tests/test-aio.c | 19 +-- thread-pool.c | 14 +- util/Makefile.objs | 2 +- util/lockcnt.c | 123 ++++++++++++++++ 37 files changed, 933 insertions(+), 267 deletions(-) create mode 100644 docs/lockcnt.txt create mode 100644 util/lockcnt.c -- 2.4.3
