Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-23 Thread Hanna Czenczek
On 02.01.24 16:24, Hanna Czenczek wrote: [...] I’ve attached the preliminary patch that I didn’t get to send (or test much) last year.  Not sure if it has the same CPU-usage-spike issue Fiona was seeing, the only functional difference is that I notify the vq after attaching the notifiers

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-23 Thread Hanna Czenczek
On 23.01.24 12:12, Fiona Ebner wrote: [...] I noticed poll_set_started() is not called, because ctx->fdmon_ops->need_wait(ctx) was true, i.e. ctx->poll_disable_cnt was positive (I'm using fdmon_poll). I then found this is because of the notifier for the event vq, being attached with

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-23 Thread Hanna Czenczek
On 22.01.24 18:52, Hanna Czenczek wrote: On 22.01.24 18:41, Hanna Czenczek wrote: On 05.01.24 15:30, Fiona Ebner wrote: Am 05.01.24 um 14:43 schrieb Fiona Ebner: Am 03.01.24 um 14:35 schrieb Paolo Bonzini: On 1/3/24 12:40, Fiona Ebner wrote: I'm happy to report that I cannot reproduce the

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-23 Thread Fiona Ebner
Am 22.01.24 um 18:52 schrieb Hanna Czenczek: > On 22.01.24 18:41, Hanna Czenczek wrote: >> On 05.01.24 15:30, Fiona Ebner wrote: >>> Am 05.01.24 um 14:43 schrieb Fiona Ebner: Am 03.01.24 um 14:35 schrieb Paolo Bonzini: > On 1/3/24 12:40, Fiona Ebner wrote: >> I'm happy to report that

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-22 Thread Hanna Czenczek
On 22.01.24 18:41, Hanna Czenczek wrote: On 05.01.24 15:30, Fiona Ebner wrote: Am 05.01.24 um 14:43 schrieb Fiona Ebner: Am 03.01.24 um 14:35 schrieb Paolo Bonzini: On 1/3/24 12:40, Fiona Ebner wrote: I'm happy to report that I cannot reproduce the CPU-usage-spike issue with the patch, but I

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-22 Thread Hanna Czenczek
On 05.01.24 15:30, Fiona Ebner wrote: Am 05.01.24 um 14:43 schrieb Fiona Ebner: Am 03.01.24 um 14:35 schrieb Paolo Bonzini: On 1/3/24 12:40, Fiona Ebner wrote: I'm happy to report that I cannot reproduce the CPU-usage-spike issue with the patch, but I did run into an assertion failure when

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-05 Thread Fiona Ebner
Am 05.01.24 um 14:43 schrieb Fiona Ebner: > Am 03.01.24 um 14:35 schrieb Paolo Bonzini: >> On 1/3/24 12:40, Fiona Ebner wrote: >>> I'm happy to report that I cannot reproduce the CPU-usage-spike issue >>> with the patch, but I did run into an assertion failure when trying to >>> verify that it

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-05 Thread Fiona Ebner
Am 03.01.24 um 14:35 schrieb Paolo Bonzini: > On 1/3/24 12:40, Fiona Ebner wrote: >> I'm happy to report that I cannot reproduce the CPU-usage-spike issue >> with the patch, but I did run into an assertion failure when trying to >> verify that it fixes my original stuck-guest-IO issue. See below

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-03 Thread Paolo Bonzini
On 1/3/24 12:40, Fiona Ebner wrote: I'm happy to report that I cannot reproduce the CPU-usage-spike issue with the patch, but I did run into an assertion failure when trying to verify that it fixes my original stuck-guest-IO issue. See below for the backtrace [0]. Hanna wrote in

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-03 Thread Fiona Ebner
Am 02.01.24 um 16:24 schrieb Hanna Czenczek: > > I’ve attached the preliminary patch that I didn’t get to send (or test > much) last year.  Not sure if it has the same CPU-usage-spike issue > Fiona was seeing, the only functional difference is that I notify the vq > after attaching the notifiers

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-02 Thread Hanna Czenczek
On 02.01.24 16:53, Paolo Bonzini wrote: On Tue, Jan 2, 2024 at 4:24 PM Hanna Czenczek wrote: I’ve attached the preliminary patch that I didn’t get to send (or test much) last year. Not sure if it has the same CPU-usage-spike issue Fiona was seeing, the only functional difference is that I

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-02 Thread Paolo Bonzini
On Tue, Jan 2, 2024 at 4:24 PM Hanna Czenczek wrote: > I’ve attached the preliminary patch that I didn’t get to send (or test > much) last year. Not sure if it has the same CPU-usage-spike issue > Fiona was seeing, the only functional difference is that I notify the vq > after attaching the

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2024-01-02 Thread Hanna Czenczek
On 13.12.23 22:15, Stefan Hajnoczi wrote: Hanna and Fiona encountered a bug in aio_set_fd_handler(): there is no matching io_poll_end() call upon removing an AioHandler when io_poll_begin() was previously called. The missing io_poll_end() call leaves virtqueue notifications disabled and the

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-19 Thread Fiona Ebner
Am 18.12.23 um 15:49 schrieb Paolo Bonzini: > On Mon, Dec 18, 2023 at 1:41 PM Fiona Ebner wrote: >> I think it's because of nested drains, because when additionally >> checking that the drain count is zero and only executing the loop then, >> that issue doesn't seem to manifest > > But isn't

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-18 Thread Paolo Bonzini
On Mon, Dec 18, 2023 at 1:41 PM Fiona Ebner wrote: > I think it's because of nested drains, because when additionally > checking that the drain count is zero and only executing the loop then, > that issue doesn't seem to manifest But isn't virtio_scsi_drained_end only run if bus->drain_count ==

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-18 Thread Stefan Hajnoczi
On Mon, Dec 18, 2023 at 01:41:38PM +0100, Fiona Ebner wrote: > Am 14.12.23 um 20:53 schrieb Stefan Hajnoczi: > > > > I will still try the other approach that Hanna and Paolo have suggested. > > It seems more palatable. I will send a v2. > > > > FYI, what I already tried downstream (for VirtIO

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-18 Thread Fiona Ebner
Am 14.12.23 um 20:53 schrieb Stefan Hajnoczi: > > I will still try the other approach that Hanna and Paolo have suggested. > It seems more palatable. I will send a v2. > FYI, what I already tried downstream (for VirtIO SCSI): > diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c > index

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-14 Thread Stefan Hajnoczi
On Thu, Dec 14, 2023 at 02:38:47PM +0100, Fiona Ebner wrote: > Am 13.12.23 um 22:15 schrieb Stefan Hajnoczi: > > But there you have it. Please let me know what you think and try your > > reproducers to see if this fixes the missing io_poll_end() issue. Thanks! > > > > Thanks to you! I applied

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-14 Thread Stefan Hajnoczi
On Thu, Dec 14, 2023 at 12:10:32AM +0100, Paolo Bonzini wrote: > On Wed, Dec 13, 2023 at 10:15 PM Stefan Hajnoczi wrote: > > Alternatives welcome! (A cleaner version of this approach might be to forbid > > cross-thread aio_set_fd_handler() calls and to refactor all > > aio_set_fd_handler()

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-14 Thread Fiona Ebner
Am 13.12.23 um 22:15 schrieb Stefan Hajnoczi: > But there you have it. Please let me know what you think and try your > reproducers to see if this fixes the missing io_poll_end() issue. Thanks! > Thanks to you! I applied the RFC (and the series it depends on) on top of 8.2.0-rc4 and this fixes

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-13 Thread Paolo Bonzini
On Wed, Dec 13, 2023 at 10:15 PM Stefan Hajnoczi wrote: > Alternatives welcome! (A cleaner version of this approach might be to forbid > cross-thread aio_set_fd_handler() calls and to refactor all > aio_set_fd_handler() callers so they come from the AioContext's home thread. > I'm starting to

Re: [RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-13 Thread Stefan Hajnoczi
Based-on: 20231205182011.1976568-1-stefa...@redhat.com (https://lore.kernel.org/qemu-devel/20231205182011.1976568-1-stefa...@redhat.com/)

[RFC 0/3] aio-posix: call ->poll_end() when removing AioHandler

2023-12-13 Thread Stefan Hajnoczi
Hanna and Fiona encountered a bug in aio_set_fd_handler(): there is no matching io_poll_end() call upon removing an AioHandler when io_poll_begin() was previously called. The missing io_poll_end() call leaves virtqueue notifications disabled and the virtqueue's ioeventfd will never become readable