This patch set aims to allow ublk server threads to better balance load amongst themselves by decoupling server threads from ublk queues/hctxs, so that multiple threads can service I/Os from a single hctx.
The first patch is the functional change in the driver which switches from per-queue daemons to per-io daemons and allows for ublk servers to balance load that is imbalanced among queues. The second patch fixes a bug in tag allocation (in the sbitmap layer) that was observed while developing a test for this feature. The next five patches add support in the selftests ublk server (kublk) for this feature, and add a test which shows the new feature working as intended. The last patch documents the new feature. Signed-off-by: Uday Shankar <ushan...@purestorage.com> --- Changes in v6: - Add a feature flag for this feature, called UBLK_F_RR_TAGS (Ming Lei) - Add test for this feature (Ming Lei) - Add documentation for this feature (Ming Lei) - Link to v5: https://lore.kernel.org/r/20250416-ublk_task_per_io-v5-0-9261ad7bf...@purestorage.com Changes in v5: - Set io->task before ublk_mark_io_ready (Caleb Sander Mateos) - Set io->task atomically, read it atomically when needed - Return 0 on success from command-specific helpers in __ublk_ch_uring_cmd (Caleb Sander Mateos) - Rename ublk_handle_need_get_data to ublk_get_data (Caleb Sander Mateos) - Link to v4: https://lore.kernel.org/r/20250415-ublk_task_per_io-v4-0-54210b91a...@purestorage.com Changes in v4: - Drop "ublk: properly serialize all FETCH_REQs" since Ming is taking it in another set - Prevent data races by marking data structures which should be read-only in the I/O path as const (Ming Lei) - Link to v3: https://lore.kernel.org/r/20250410-ublk_task_per_io-v3-0-b811e8f45...@purestorage.com Changes in v3: - Check for UBLK_IO_FLAG_ACTIVE on I/O again after taking lock to ensure that two concurrent FETCH_REQs on the same I/O can't succeed (Caleb Sander Mateos) - Link to v2: https://lore.kernel.org/r/20250408-ublk_task_per_io-v2-0-b97877e6f...@purestorage.com Changes in v2: - Remove changes split into other patches - To ease error handling/synchronization, associate each I/O (instead of each queue) to the last task that issues a FETCH_REQ against it. Only that task is allowed to operate on the I/O. - Link to v1: https://lore.kernel.org/r/20241002224437.3088981-1-ushan...@purestorage.com --- Uday Shankar (8): ublk: have a per-io daemon instead of a per-queue daemon sbitmap: fix off-by-one when wrapping hint selftests: ublk: kublk: plumb q_id in io_uring user_data selftests: ublk: kublk: tie sqe allocation to io instead of queue selftests: ublk: kublk: lift queue initialization out of thread selftests: ublk: kublk: move per-thread data out of ublk_queue selftests: ublk: kublk: decouple ublk_queues from ublk server threads Documentation: ublk: document UBLK_F_RR_TAGS Documentation/block/ublk.rst | 83 +++++- drivers/block/ublk_drv.c | 82 ++--- include/uapi/linux/ublk_cmd.h | 8 + lib/sbitmap.c | 4 +- tools/testing/selftests/ublk/Makefile | 1 + tools/testing/selftests/ublk/fault_inject.c | 4 +- tools/testing/selftests/ublk/file_backed.c | 20 +- tools/testing/selftests/ublk/kublk.c | 329 ++++++++++++++------- tools/testing/selftests/ublk/kublk.h | 73 +++-- tools/testing/selftests/ublk/null.c | 12 +- tools/testing/selftests/ublk/stripe.c | 17 +- tools/testing/selftests/ublk/test_generic_08.sh | 61 ++++ .../selftests/ublk/trace/count_ios_per_tid.bt | 9 + 13 files changed, 488 insertions(+), 215 deletions(-) --- base-commit: 037af793557ed192b2c10cf2379ac97abacedf55 change-id: 20250408-ublk_task_per_io-c693cf608d7a Best regards, -- Uday Shankar <ushan...@purestorage.com>
