Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Jinhao Fan
After digging through the source code, I found event_notifier_cleanup() only closes the eventfd, but does not de-register the event from QEMU’s event loop. event_notifier_set_handler() actually interacts with the event loop. It is a wrapper around aio_set_event_notifier(), which is again a wrapper

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Klaus Jensen
On Jul 27 16:16, Jinhao Fan wrote: > at 3:06 PM, Klaus Jensen wrote: > > > On Jul 26 14:08, Klaus Jensen wrote: > >> Alright. Forget about the iommu, that was just a coincidence. > >> > >> This patch seems to fix it. I guess it is the > >> event_notifier_set_handler(..., NULL) that does the

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Jinhao Fan
at 3:06 PM, Klaus Jensen wrote: > On Jul 26 14:08, Klaus Jensen wrote: >> Alright. Forget about the iommu, that was just a coincidence. >> >> This patch seems to fix it. I guess it is the >> event_notifier_set_handler(..., NULL) that does the trick, but I'd like >> to understand why ;) >> >>

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-27 Thread Klaus Jensen
On Jul 26 14:08, Klaus Jensen wrote: > > Alright. Forget about the iommu, that was just a coincidence. > > This patch seems to fix it. I guess it is the > event_notifier_set_handler(..., NULL) that does the trick, but I'd like > to understand why ;) > > > diff --git i/hw/nvme/ctrl.c

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Stefan Hajnoczi
On Tue, 5 Jul 2022 at 10:25, Jinhao Fan wrote: > > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 13:32, Klaus Jensen wrote: > On Jul 26 13:24, Klaus Jensen wrote: > > On Jul 26 12:09, Klaus Jensen wrote: > > > On Jul 26 11:19, Klaus Jensen wrote: > > > > On Jul 26 15:55, Jinhao Fan wrote: > > > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > > > > > On Jul 26 15:35, Jinhao Fan

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 13:24, Klaus Jensen wrote: > On Jul 26 12:09, Klaus Jensen wrote: > > On Jul 26 11:19, Klaus Jensen wrote: > > > On Jul 26 15:55, Jinhao Fan wrote: > > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > > > >> at 4:55 AM, Klaus Jensen wrote: >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 12:09, Klaus Jensen wrote: > On Jul 26 11:19, Klaus Jensen wrote: > > On Jul 26 15:55, Jinhao Fan wrote: > > > at 3:41 PM, Klaus Jensen wrote: > > > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > > >> at 4:55 AM, Klaus Jensen wrote: > > > >> > > > >>> We have a regression following

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 11:19, Klaus Jensen wrote: > On Jul 26 15:55, Jinhao Fan wrote: > > at 3:41 PM, Klaus Jensen wrote: > > > > > On Jul 26 15:35, Jinhao Fan wrote: > > >> at 4:55 AM, Klaus Jensen wrote: > > >> > > >>> We have a regression following this patch that we need to address. > > >>> > > >>>

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 15:55, Jinhao Fan wrote: > at 3:41 PM, Klaus Jensen wrote: > > > On Jul 26 15:35, Jinhao Fan wrote: > >> at 4:55 AM, Klaus Jensen wrote: > >> > >>> We have a regression following this patch that we need to address. > >>> > >>> With this patch, issuing a reset on the device (`nvme

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Jinhao Fan
at 3:41 PM, Klaus Jensen wrote: > On Jul 26 15:35, Jinhao Fan wrote: >> at 4:55 AM, Klaus Jensen wrote: >> >>> We have a regression following this patch that we need to address. >>> >>> With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` >>> will do the trick) causes QEMU

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Klaus Jensen
On Jul 26 15:35, Jinhao Fan wrote: > at 4:55 AM, Klaus Jensen wrote: > > > > > We have a regression following this patch that we need to address. > > > > With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` > > will do the trick) causes QEMU to hog my host cpu at 100%. > >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-26 Thread Jinhao Fan
at 4:55 AM, Klaus Jensen wrote: > > We have a regression following this patch that we need to address. > > With this patch, issuing a reset on the device (`nvme reset /dev/nvme0` > will do the trick) causes QEMU to hog my host cpu at 100%. > > I'm still not sure what causes this. The trace

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-25 Thread Klaus Jensen
On Jul 5 22:24, Jinhao Fan wrote: > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu thread run both

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-13 Thread Klaus Jensen
On Jul 12 14:23, Klaus Jensen wrote: > On Jul 9 11:06, Jinhao Fan wrote: > > at 10:24 PM, Jinhao Fan wrote: > > > > > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, > > > const NvmeRequest *req) > > > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > > >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-12 Thread Klaus Jensen
On Jul 9 11:06, Jinhao Fan wrote: > at 10:24 PM, Jinhao Fan wrote: > > > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > > NvmeRequest *req) > > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > > uint64_t eis_addr = le64_to_cpu(req->cmd.dptr.prp2); >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-08 Thread Jinhao Fan
at 10:24 PM, Jinhao Fan wrote: > @@ -5793,6 +5891,7 @@ static uint16_t nvme_dbbuf_config(NvmeCtrl *n, const > NvmeRequest *req) > uint64_t dbs_addr = le64_to_cpu(req->cmd.dptr.prp1); > uint64_t eis_addr = le64_to_cpu(req->cmd.dptr.prp2); > int i; > +int ret; > I just noticed

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-07 Thread Jinhao Fan
at 1:51 PM, Klaus Jensen wrote: > On Jul 6 19:34, Jinhao Fan wrote: >> at 2:43 AM, Keith Busch wrote: >> >>> On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: On Jul 5 22:24, Jinhao Fan wrote: > @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Klaus Jensen
On Jul 6 19:34, Jinhao Fan wrote: > at 2:43 AM, Keith Busch wrote: > > > On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: > >> On Jul 5 22:24, Jinhao Fan wrote: > >>> @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue > >>> *cq, NvmeRequest *req) > >>> >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Jinhao Fan
at 2:43 AM, Keith Busch wrote: > On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: >> On Jul 5 22:24, Jinhao Fan wrote: >>> @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue >>> *cq, NvmeRequest *req) >>> >>> QTAILQ_REMOVE(>sq->out_req_list, req,

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-06 Thread Jinhao Fan
at 1:11 AM, Klaus Jensen wrote: > On Jul 5 22:24, Jinhao Fan wrote: >> Add property "ioeventfd" which is enabled by default. When this is >> enabled, updates on the doorbell registers will cause KVM to signal >> an event to the QEMU main loop to handle the doorbell updates. >> Therefore,

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-05 Thread Keith Busch
On Tue, Jul 05, 2022 at 07:11:36PM +0200, Klaus Jensen wrote: > On Jul 5 22:24, Jinhao Fan wrote: > > @@ -1374,7 +1374,14 @@ static void nvme_enqueue_req_completion(NvmeCQueue > > *cq, NvmeRequest *req) > > > > QTAILQ_REMOVE(>sq->out_req_list, req, entry); > >

Re: [PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-05 Thread Klaus Jensen
On Jul 5 22:24, Jinhao Fan wrote: > Add property "ioeventfd" which is enabled by default. When this is > enabled, updates on the doorbell registers will cause KVM to signal > an event to the QEMU main loop to handle the doorbell updates. > Therefore, instead of letting the vcpu thread run both

[PATCH v4] hw/nvme: Use ioeventfd to handle doorbell updates

2022-07-05 Thread Jinhao Fan
Add property "ioeventfd" which is enabled by default. When this is enabled, updates on the doorbell registers will cause KVM to signal an event to the QEMU main loop to handle the doorbell updates. Therefore, instead of letting the vcpu thread run both guest VM and IO emulation, we now use the