On Wed, Jun 04, 2025 at 03:27:52PM +0200, Hanna Czenczek wrote: > Hi, > > This series: > - Fixes some bugs/minor inconveniences, > - Removes libfuse from the request processing path, > - Make the FUSE export use coroutines for request handling, > - Introduces multi-threading into the FUSE export. > > More detail on the v1 cover letter: > > https://lists.nongnu.org/archive/html/qemu-block/2025-03/msg00359.html > > > Changes from v1: > - Patch 1: Clarified “polling” to be `aio_poll()` > - Patch 11 (new): Pulled out from patch 13 (prev. 11) > - Patch 12 (new): Suggested by Eric > - Patch 13 (prev. 11): > - Drop false polling handlers > - Use qemu_fcntl_addfl() instead of fcntl(F_SETFL) to keep > pre-existing FD flags > - Add a note that the buffer returned by read needs to be freed via > qemu_vfree() > - Pulled out a variable rename into the new patch 11 > - Patch 15 (prev. 13): > - Simplified the co_read_from_fuse_fd() interface thanks to no longer > needing to support poll handlers > - Increment in-flight counter before entering the coroutine to make it > more obvious how tihs ensures that the export pointer remains valid > throughout > - Patch 16 (new): Add a common multi-threading interface for exports > instead of a specific one just for FUSE > - Patch 17 (new): Test cases for this new interface > - Patch 18 (prev. 14): > - Drop the contrasting with virtio-blk from the commit message; > explaining the interface is no longer necessary now that it’s > introduced separately in patch 16. > - Generally, the interface definition is removed in favor of the new > one in patch 16. > - Some rebase conflicts (due to other changes earlier in this series). > - Patch 19 (new): Stefan suggested adding an explicit note for users on > how multi-threading behaves with FUSE, not least because in the future > this behavior may depend on the specific implementation features > chosen (io-uring or not). Because the actual multi-thread interface > is now on the common export options, it is no longer obvious where to > put this implementation note; I decided to put it into the general > description of the FUSE export options, inside of this dedicated > patch. > - Patch 20 (new): Simple sanity test for FUSE multi-threading (just test > that e.g. nothing crashes when running qemu-img bench on top) > - Patch 21 (prev. 15): Rebase conflict due to the changes in patch 15; > kept Stefan’s R-b anyway > > > Review notes/suggestions I deliberately did not follow in v2: > - Stefan suggested to make patch 1 simpler and more robust by allocating > a new buffer for each request. This is indeed a simple change (for > patch 1) that I wouldn’t mind, and that I also started to implement. > However, eventually I decided against it: > The problem doesn’t disappear with the rest of the series, it > basically stays the exact same; though instead of an implicit > aio_poll() leading to nested polling, it turns into an implicit > coroutine yield doing pretty much the same. > For performance, it is better not to allocate a new buffer for each > request; we only really need a bounce buffer for writes, as there is > no other case where we’d continue to read the request buffer after > yielding. Therefore, the final state I would like to have after this > series is to use a common request buffer for all requests on a single > queue, only using a bounce buffer for writes. > With that, I think it’s better to implement exactly that right from > the start, rather than introducing a new intermediate state. > > > git-backport-diff from v1: > > Key: > [----] : patches are identical > [####] : number of functional differences between upstream/downstream patch > [down] : patch is downstream-only > The flags [FC] indicate (F)unctional and (C)ontextual differences, > respectively > > 001/21:[0012] [FC] 'fuse: Copy write buffer content before polling' > 002/21:[----] [--] 'fuse: Ensure init clean-up even with error_fatal' > 003/21:[----] [--] 'fuse: Remove superfluous empty line' > 004/21:[----] [--] 'fuse: Explicitly set inode ID to 1' > 005/21:[----] [--] 'fuse: Change setup_... to mount_fuse_export()' > 006/21:[----] [--] 'fuse: Fix mount options' > 007/21:[----] [--] 'fuse: Set direct_io and parallel_direct_writes' > 008/21:[----] [--] 'fuse: Introduce fuse_{at,de}tach_handlers()' > 009/21:[----] [--] 'fuse: Introduce fuse_{inc,dec}_in_flight()' > 010/21:[----] [--] 'fuse: Add halted flag' > 011/21:[down] 'fuse: Rename length to blk_len in fuse_write()' > 012/21:[down] 'block: Move qemu_fcntl_addfl() into osdep.c' > 013/21:[0077] [FC] 'fuse: Manually process requests (without libfuse)' > 014/21:[----] [--] 'fuse: Reduce max read size' > 015/21:[0061] [FC] 'fuse: Process requests in coroutines' > 016/21:[down] 'block/export: Add multi-threading interface' > 017/21:[down] 'iotests/307: Test multi-thread export interface' > 018/21:[0077] [FC] 'fuse: Implement multi-threading' > 019/21:[down] 'qapi/block-export: Document FUSE's multi-threading' > 020/21:[down] 'iotests/308: Add multi-threading sanity test' > 021/21:[0002] [FC] 'fuse: Increase MAX_WRITE_SIZE with a second buffer' > > > Hanna Czenczek (21): > fuse: Copy write buffer content before polling > fuse: Ensure init clean-up even with error_fatal > fuse: Remove superfluous empty line > fuse: Explicitly set inode ID to 1 > fuse: Change setup_... to mount_fuse_export() > fuse: Fix mount options > fuse: Set direct_io and parallel_direct_writes > fuse: Introduce fuse_{at,de}tach_handlers() > fuse: Introduce fuse_{inc,dec}_in_flight() > fuse: Add halted flag > fuse: Rename length to blk_len in fuse_write() > block: Move qemu_fcntl_addfl() into osdep.c > fuse: Manually process requests (without libfuse) > fuse: Reduce max read size > fuse: Process requests in coroutines > block/export: Add multi-threading interface > iotests/307: Test multi-thread export interface > fuse: Implement multi-threading > qapi/block-export: Document FUSE's multi-threading > iotests/308: Add multi-threading sanity test > fuse: Increase MAX_WRITE_SIZE with a second buffer > > qapi/block-export.json | 39 +- > include/block/export.h | 12 +- > include/qemu/osdep.h | 1 + > block/export/export.c | 48 +- > block/export/fuse.c | 1181 ++++++++++++++++++++------ > block/export/vduse-blk.c | 7 + > block/export/vhost-user-blk-server.c | 8 + > block/file-posix.c | 17 +- > nbd/server.c | 6 + > util/osdep.c | 18 + > tests/qemu-iotests/307 | 47 + > tests/qemu-iotests/307.out | 18 + > tests/qemu-iotests/308 | 55 +- > tests/qemu-iotests/308.out | 61 +- > 14 files changed, 1213 insertions(+), 305 deletions(-) > > -- > 2.49.0 >
Nice, looks like there are some goodies in here that Brian can use in his FUSE-over-io_uring project, like the new iothread=['iothread0', 'iothread1', ...] syntax for specifying multiple IOThreads. Reviewed-by: Stefan Hajnoczi <stefa...@redhat.com>
signature.asc
Description: PGP signature
