Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-07 Thread Si-Wei Liu
On 5/1/2024 11:44 PM, Eugenio Perez Martin wrote: On Thu, May 2, 2024 at 1:16 AM Si-Wei Liu wrote: On 4/30/2024 10:19 AM, Eugenio Perez Martin wrote: On Tue, Apr 30, 2024 at 7:55 AM Si-Wei Liu wrote: On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-07 Thread Si-Wei Liu
On 5/1/2024 11:18 PM, Eugenio Perez Martin wrote: On Thu, May 2, 2024 at 12:09 AM Si-Wei Liu wrote: On 4/30/2024 11:11 AM, Eugenio Perez Martin wrote: On Mon, Apr 29, 2024 at 1:19 PM Jonah Palmer wrote: On 4/29/24 4:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-01 Thread Si-Wei Liu
On 4/30/2024 10:19 AM, Eugenio Perez Martin wrote: On Tue, Apr 30, 2024 at 7:55 AM Si-Wei Liu wrote: On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-05-01 Thread Si-Wei Liu
On 4/30/2024 11:11 AM, Eugenio Perez Martin wrote: On Mon, Apr 29, 2024 at 1:19 PM Jonah Palmer wrote: On 4/29/24 4:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-29 Thread Si-Wei Liu
On 4/29/2024 1:14 AM, Eugenio Perez Martin wrote: On Thu, Apr 25, 2024 at 7:44 PM Si-Wei Liu wrote: On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM Si-Wei Liu wrote: On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-25 Thread Si-Wei Liu
On 4/24/2024 12:33 AM, Eugenio Perez Martin wrote: On Wed, Apr 24, 2024 at 12:21 AM Si-Wei Liu wrote: On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM Si-Wei Liu wrote: On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-23 Thread Si-Wei Liu
On 4/22/2024 1:49 AM, Eugenio Perez Martin wrote: On Sat, Apr 20, 2024 at 1:50 AM Si-Wei Liu wrote: On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM Si-Wei Liu wrote: On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-19 Thread Si-Wei Liu
On 4/19/2024 1:29 AM, Eugenio Perez Martin wrote: On Thu, Apr 18, 2024 at 10:46 PM Si-Wei Liu wrote: On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track the mappings of virtio-net shadow virtqueue. This mappings may not match with the GPA->HVA ones. This cau

Re: [RFC 1/2] iova_tree: add an id member to DMAMap

2024-04-18 Thread Si-Wei Liu
On 4/10/2024 3:03 AM, Eugenio Pérez wrote: IOVA tree is also used to track the mappings of virtio-net shadow virtqueue. This mappings may not match with the GPA->HVA ones. This causes a problem when overlapped regions (different GPA but same translated HVA) exists in the tree, as looking

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-04-03 Thread Si-Wei Liu
On 4/2/2024 5:01 AM, Eugenio Perez Martin wrote: On Tue, Apr 2, 2024 at 8:19 AM Si-Wei Liu wrote: On 2/14/2024 11:11 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 7:29 PM Si-Wei Liu wrote: Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-04-02 Thread Si-Wei Liu
On 2/14/2024 11:11 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 7:29 PM Si-Wei Liu wrote: Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me

Re: [External] : Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-25 Thread Si-Wei Liu
On 3/24/2024 11:13 PM, Jason Wang wrote: On Sat, Mar 23, 2024 at 5:14 AM Si-Wei Liu wrote: On 3/21/2024 10:08 PM, Jason Wang wrote: On Fri, Mar 22, 2024 at 5:43 AM Si-Wei Liu wrote: On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-22 Thread Si-Wei Liu
On 3/21/2024 10:08 PM, Jason Wang wrote: On Fri, Mar 22, 2024 at 5:43 AM Si-Wei Liu wrote: On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-21 Thread Si-Wei Liu
On 3/20/2024 8:56 PM, Jason Wang wrote: On Thu, Mar 21, 2024 at 5:03 AM Si-Wei Liu wrote: On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-20 Thread Si-Wei Liu
On 3/19/2024 8:27 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:16 AM Si-Wei Liu wrote: On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-20 Thread Si-Wei Liu
On 3/19/2024 8:25 PM, Jason Wang wrote: On Tue, Mar 19, 2024 at 6:06 AM Si-Wei Liu wrote: On 3/17/2024 8:20 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:33 AM Si-Wei Liu wrote: On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-18 Thread Si-Wei Liu
On 3/17/2024 8:22 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:45 AM Si-Wei Liu wrote: On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-18 Thread Si-Wei Liu
On 3/17/2024 8:20 PM, Jason Wang wrote: On Sat, Mar 16, 2024 at 2:33 AM Si-Wei Liu wrote: On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate

Re: [PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-15 Thread Si-Wei Liu
On 3/14/2024 9:03 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost the bigger the number amount of queues are set up e.g. on idle guests migration the following

Re: [PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-15 Thread Si-Wei Liu
On 3/14/2024 8:50 PM, Jason Wang wrote: On Fri, Mar 15, 2024 at 5:39 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate vhost loggers for the specific vhost type have to be used. Make the vhost logger per backend

[PATCH v4 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
eues -> 8.71%[.] vhost_dev_sync_region.isra.13 2 devices, 8 queues -> 7.97% [.] vhost_dev_sync_region.isra.14 Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- v3 -> v4: - add comment to clarify effect on cache locality and performance v2

[PATCH v4 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
-by: Si-Wei Liu --- v3->v4: - remove checking NULL return value from vhost_log_get v2->v3: - remove non-effective assertion that never be reached - do not return NULL from vhost_log_get() - add neccessary assertions to vhost_log_get() --- hw/virtio/vhost.

Re: [PATCH v3 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
On 3/14/2024 8:25 AM, Eugenio Perez Martin wrote: On Thu, Mar 14, 2024 at 9:38 AM Si-Wei Liu wrote: There could be a mix of both vhost-user and vhost-kernel clients in the same QEMU process, where separate vhost loggers for the specific vhost type have to be used. Make the vhost logger per

Re: [PATCH v3 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
On 3/14/2024 8:34 AM, Eugenio Perez Martin wrote: On Thu, Mar 14, 2024 at 9:38 AM Si-Wei Liu wrote: On setups with one or more virtio-net devices with vhost on, dirty tracking iteration increases cost the bigger the number amount of queues are set up e.g. on idle guests migration

[PATCH v3 1/2] vhost: dirty log should be per backend type

2024-03-14 Thread Si-Wei Liu
-by: Si-Wei Liu --- v2->v3: - remove non-effective assertion that never be reached - do not return NULL from vhost_log_get() - add neccessary assertions to vhost_log_get() --- hw/virtio/vhost.c | 50 ++ 1 file changed, 38 insertions(+),

[PATCH v3 2/2] vhost: Perform memory section dirty scans once per iteration

2024-03-14 Thread Si-Wei Liu
eues -> 8.71%[.] vhost_dev_sync_region.isra.13 2 devices, 8 queues -> 7.97% [.] vhost_dev_sync_region.isra.14 Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- v2 -> v3: - add after-fix benchmark to commit log - rename vhost_log_dev_enabled to

Re: [PATCH v2 1/2] vhost: dirty log should be per backend type

2024-03-13 Thread Si-Wei Liu
On 3/12/2024 8:07 AM, Michael S. Tsirkin wrote: On Wed, Feb 14, 2024 at 10:42:29AM -0800, Si-Wei Liu wrote: Hi Michael, I'm taking off for 2+ weeks, but please feel free to provide comment and feedback while I'm off. I'll be checking emails still, and am about to address any opens as soon

Re: [PATCH 12/12] vdpa: fix network breakage after cancelling migration

2024-03-13 Thread Si-Wei Liu
On 3/13/2024 11:12 AM, Michael Tokarev wrote: 14.02.2024 14:28, Si-Wei Liu wrote: Fix an issue where cancellation of ongoing migration ends up with no network connectivity. When canceling migration, SVQ will be switched back to the passthrough mode, but the right call fd is not programed

Re: [PATCH 04/12] vdpa: factor out vhost_vdpa_net_get_nc_vdpa

2024-02-14 Thread Si-Wei Liu
On 2/14/2024 10:54 AM, Eugenio Perez Martin wrote: On Wed, Feb 14, 2024 at 1:39 PM Si-Wei Liu wrote: Introduce new API. No functional change on existing API. Acked-by: Jason Wang Signed-off-by: Si-Wei Liu I'm ok with the new function, but doesn't the compiler complain because adding

Re: [PATCH v2 1/2] vhost: dirty log should be per backend type

2024-02-14 Thread Si-Wei Liu
Hi Michael, I'm taking off for 2+ weeks, but please feel free to provide comment and feedback while I'm off. I'll be checking emails still, and am about to address any opens as soon as I am back. Thanks, -Siwei On 2/14/2024 3:50 AM, Si-Wei Liu wrote: There could be a mix of both vhost-user

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-14 Thread Si-Wei Liu
On 2/13/2024 8:26 AM, Eugenio Perez Martin wrote: On Tue, Feb 13, 2024 at 11:22 AM Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me and the original issue I saw with x-svq=on should be gone. However, after

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-14 Thread Si-Wei Liu
Hi Michael, On 2/13/2024 2:22 AM, Michael S. Tsirkin wrote: On Mon, Feb 05, 2024 at 05:10:36PM -0800, Si-Wei Liu wrote: Hi Eugenio, I thought this new code looks good to me and the original issue I saw with x-svq=on should be gone. However, after rebase my tree on top of this, there's a new

[PATCH v2 2/2] vhost: Perform memory section dirty scans once per iteration

2024-02-14 Thread Si-Wei Liu
ogger device goes away for some reason, the logger will be re-selected from the rest of vhost devices. Co-developed-by: Joao Martins Signed-off-by: Joao Martins Signed-off-by: Si-Wei Liu --- hw/virtio/vhost.c | 75 +++ include/hw/virtio/vhost

[PATCH v2 1/2] vhost: dirty log should be per backend type

2024-02-14 Thread Si-Wei Liu
-by: Si-Wei Liu --- hw/virtio/vhost.c | 49 + 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 2c9ac79..ef6d9b5 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -43,8 +43,8 @@ do

[PATCH 07/12] vdpa: add vhost_vdpa_set_dev_vring_base trace for svq mode

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 5 - 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace

[PATCH 04/12] vdpa: factor out vhost_vdpa_net_get_nc_vdpa

2024-02-14 Thread Si-Wei Liu
Introduce new API. No functional change on existing API. Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 06c83b4..4168cad 100644 --- a/net/vhost

[PATCH 08/12] vdpa: add trace events for vhost_vdpa_net_load_cmd

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 2 ++ net/vhost-vdpa.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/net/trace-events b/net/trace-events index aab666a..88f56f2 100644 --- a/net/trace-events +++ b

[PATCH 10/12] vdpa: define SVQ transitioning state for mode switching

2024-02-14 Thread Si-Wei Liu
Will be used in following patches. DISABLING(-1) means SVQ is being switched off to passthrough mode. ENABLING(1) means passthrough VQs are being switched to SVQ. DONE(0) means SVQ switching is completed. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 9 + 1 file

[PATCH 01/12] vdpa: add back vhost_vdpa_net_first_nc_vdpa

2024-02-14 Thread Si-Wei Liu
Previous commits had it removed. Now adding it back because this function will be needed by future patches. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net

[PATCH 09/12] vdpa: add trace event for vhost_vdpa_net_load_mq

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 1 + net/vhost-vdpa.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/net/trace-events b/net/trace-events index 88f56f2..cda960f 100644 --- a/net/trace-events +++ b

[PATCH 06/12] vdpa: add vhost_vdpa_get_vring_base trace for svq mode

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace

[PATCH 03/12] vdpa: factor out vhost_vdpa_last_dev

2024-02-14 Thread Si-Wei Liu
Generalize duplicated condition check for the last vq of vdpa device to a common function. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b

[PATCH 05/12] vdpa: add vhost_vdpa_set_address_space_id trace

2024-02-14 Thread Si-Wei Liu
For better debuggability and observability. Reviewed-by: Eugenio Pérez Signed-off-by: Si-Wei Liu --- net/trace-events | 3 +++ net/vhost-vdpa.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/net/trace-events b/net/trace-events index 823a071..aab666a 100644 --- a/net/trace-events +++ b

[PATCH 11/12] vdpa: indicate transitional state for SVQ switching

2024-02-14 Thread Si-Wei Liu
svq_switching indicates the transitional state whether or not SVQ mode switching is in progress, and towards which direction. Add the neccessary state around where the switching would take place. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +++ 1 file changed, 3 insertions(+) diff --git

[PATCH 12/12] vdpa: fix network breakage after cancelling migration

2024-02-14 Thread Si-Wei Liu
period, the shadow_vqs_enabled hadn't been set back to false yet, causing the installation of call fd inadvertently bypassed. Fixes: a8ac88585da1 ("vhost: Add Shadow VirtQueue call forwarding capabilities") Cc: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- hw/vi

[PATCH 02/12] vdpa: no repeat setting shadow_data

2024-02-14 Thread Si-Wei Liu
space. Reviewed-by: Eugenio Pérez Acked-by: Jason Wang Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 4479ffa..06c83b4 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -354,13

[PATCH 00/12] Preparatory patches for live migration downtime improvement

2024-02-14 Thread Si-Wei Liu
://lore.kernel.org/qemu-devel/1701970793-6865-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (12): vdpa: add back vhost_vdpa_net_first_nc_vdpa vdpa: no repeat setting shadow_data vdpa: factor out vhost_vdpa_last_dev vdpa: factor out vhost_vdpa_net_get_nc_vdpa vdpa: add

Re: [PATCH v2 6/7] vdpa: move iova_tree allocation to net_vhost_vdpa_init

2024-02-05 Thread Si-Wei Liu
its dependencies to the initialization too. In particular devices with x-svq=on need a valid iova_tree from the beginning. Simplify the code also consolidating the two creation points: the first data vq in case of SVQ active and CVQ start in case only CVQ uses it. Suggested-by: Si-Wei Liu Signed

Re: [PATCH 1/6] vdpa: check for iova tree initialized at net_client_start

2024-01-31 Thread Si-Wei Liu
Hi Eugenio, Maybe there's some patch missing, but I saw this core dump when x-svq=on is specified while waiting for the incoming migration on destination host: (gdb) bt #0  0x5643b24cc13c in vhost_iova_tree_map_alloc (tree=0x0, map=map@entry=0x7ffd58c54830) at

[PATCH 31/40] vdpa: batch map and unmap around cvq svq start/stop

2023-12-07 Thread Si-Wei Liu
Coalesce map or unmap operations to exact one DMA batch to reduce potential impact on performance. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index bc72345..1c1d61f 100644

[PATCH 28/40] vdpa: support iotlb_batch_asid

2023-12-07 Thread Si-Wei Liu
-Wei Liu --- hw/virtio/vhost-vdpa.c | 25 +++-- include/hw/virtio/vhost-vdpa.h | 1 + net/vhost-vdpa.c | 1 + 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index d3f5721..b7896a8 100644

[PATCH 40/40] vdpa: add trace event for vhost_vdpa_net_load_mq

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 1 + net/vhost-vdpa.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/net/trace-events b/net/trace-events index be087e6..c128cc4 100644 --- a/net/trace-events +++ b/net/trace-events @@ -30,3

[PATCH 07/40] vdpa: move around vhost_vdpa_set_address_space_id

2023-12-07 Thread Si-Wei Liu
Move it a few lines ahead to make function call easier for those before it. No funtional change involved. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 36 ++-- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost

[PATCH 22/40] vdpa: factor out vhost_vdpa_map_batch_begin

2023-12-07 Thread Si-Wei Liu
Refactoring only. No functional change. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 25 - 2 files changed, 17 insertions(+), 10 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 9725d44..b0239b8

[PATCH 12/40] vdpa: check map_thread_enabled before join maps thread

2023-12-07 Thread Si-Wei Liu
The next patches will also register memory listener on demand, hence the need to differentiate the map_thread case from the rest. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost

[PATCH 06/40] vhost: make svq work with gpa without iova translation

2023-12-07 Thread Si-Wei Liu
, with dedicated or isolated address space for SVQ descriptors, the IOVA is exactly same as the guest GPA space where translation would not be needed any more. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-shadow-virtqueue.c | 35 +++ 1 file changed, 23 insertions(+), 12

[PATCH 04/40] vdpa: piggyback desc_group index when probing isolated cvq

2023-12-07 Thread Si-Wei Liu
Same as the previous commit, but do it for cvq instead of data vqs. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 21 + 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 0cf3147..cb5705d 100644 --- a/net/vhost-vdpa.c

[PATCH 25/40] vdpa: add asid to dma_batch_once API

2023-12-07 Thread Si-Wei Liu
So that DMA batching API can operate on other ASID than 0. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 4 ++-- hw/virtio/vhost-vdpa.c | 14 -- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 3411a07

[PATCH 35/40] vdpa: add vhost_vdpa_set_address_space_id trace

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 3 +++ net/vhost-vdpa.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/net/trace-events b/net/trace-events index 823a071..aab666a 100644 --- a/net/trace-events +++ b/net/trace-events @@ -23,3

[PATCH 21/40] vdpa: vhost_vdpa_dma_batch_end_once rename

2023-12-07 Thread Si-Wei Liu
No functional changes. Rename only. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 47c764b..013bfa2 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost

[PATCH 14/40] vdpa: convert iova_tree to ref count based

2023-12-07 Thread Si-Wei Liu
So that it can be freed from vhost_vdpa_cleanup on the last deref. The next few patches will try to make iova tree life cycle not depend on memory listener, and there's possiblity to keep iova tree around when memory mapping is not changed across device reset. Signed-off-by: Si-Wei Liu --- net

[PATCH 11/40] vdpa: factor out vhost_vdpa_last_dev

2023-12-07 Thread Si-Wei Liu
Generalize duplicated condition check for the last vq of vdpa device to a common function. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 30dff95..2b1cc14

[PATCH 15/40] vdpa: add svq_switching and flush_map to header

2023-12-07 Thread Si-Wei Liu
Will be used in next patches. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 7b8d3bf..0fe0f60 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include

[PATCH 34/40] vdpa: fix network breakage after cancelling migration

2023-12-07 Thread Si-Wei Liu
period, the shadow_vqs_enabled hadn't been set back to false yet, causing the installation of call fd inadvertently bypassed. Fixes: a8ac88585da1 ("vhost: Add Shadow VirtQueue call forwarding capabilities") Cc: Eugenio Pérez Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 7

[PATCH 09/40] vdpa: no repeat setting shadow_data

2023-12-07 Thread Si-Wei Liu
space. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index c9bfc6f..2555897 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -387,13 +387,12 @@ static int vhost_vdpa_net_data_start

[PATCH 13/40] vdpa: ref counting VhostVDPAShared

2023-12-07 Thread Si-Wei Liu
be easy to free resource upon the last deref. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 2 ++ net/vhost-vdpa.c | 14 ++ 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index

[PATCH 30/40] vdpa: batch map/unmap op per svq pair basis

2023-12-07 Thread Si-Wei Liu
Coalesce multiple map or unmap operations to just one so that all mapping setup or teardown can occur in a single DMA batch. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 5 + 1 file changed, 5 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index

[PATCH 18/40] vdpa: unregister listener on last dev cleanup

2023-12-07 Thread Si-Wei Liu
So that the free of iova tree struct can be safely deferred to until the last vq referencing it goes away. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 4f026db

[PATCH 17/40] vdpa: judge if map can be kept across reset

2023-12-07 Thread Si-Wei Liu
and cvq isolation capability. Have to hook the evaluation function to NetClient's .poll op as .vhost_reset_status runs ahead of .stop, and .vhost_dev_start don't have access to the vhost-vdpa net's information. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 40

[PATCH 38/40] vdpa: add trace events for eval_flush

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 2 ++ net/vhost-vdpa.c | 7 +++ 2 files changed, 9 insertions(+) diff --git a/net/trace-events b/net/trace-events index aab666a..d650c71 100644 --- a/net/trace-events +++ b/net/trace-events

[PATCH 33/40] vdpa: batch multiple dma_unmap to a single call for vm stop

2023-12-07 Thread Si-Wei Liu
idx: 1 109531@1693367277.014753:vhost_vdpa_get_vq_index dev: 0x55c9339262e0 idx: 2 vq idx: 2 109531@1693367277.014756:vhost_vdpa_get_vq_index dev: 0x55c9339262e0 idx: 3 vq idx: 3 Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 7 +-- include/hw/virtio/vhost-vdpa.h | 3

[PATCH 39/40] vdpa: add trace events for vhost_vdpa_net_load_cmd

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- net/trace-events | 2 ++ net/vhost-vdpa.c | 2 ++ 2 files changed, 4 insertions(+) diff --git a/net/trace-events b/net/trace-events index d650c71..be087e6 100644 --- a/net/trace-events +++ b/net/trace-events @@ -28,3

[PATCH 29/40] vdpa: expose API vhost_vdpa_dma_batch_once

2023-12-07 Thread Si-Wei Liu
So that the batching API can be called from other file externally than the local. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 21 +++-- include/hw/virtio/vhost-vdpa.h | 3 +++ 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/hw/virtio/vhost

[PATCH 20/40] vdpa: avoid mapping flush across reset

2023-12-07 Thread Si-Wei Liu
, it would save substantial time from pinning and mapping unneccessarily when moving descriptors on to or out of shadow mode. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio

[PATCH 19/40] vdpa: should avoid map flushing with persistent iotlb

2023-12-07 Thread Si-Wei Liu
that the next patch will be able to use it to keep the maps. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 1 + hw/virtio/vhost-vdpa.c | 20 2 files changed, 21 insertions(+) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 77905d1..9725d44 100644 --- a/hw

[PATCH 26/40] vdpa: return int for dma_batch_once API

2023-12-07 Thread Si-Wei Liu
Return zero for success for now. Prepare for non-zero return in the next few patches. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 14 +- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 2db2832..e0137f0

[PATCH 03/40] vdpa: probe descriptor group index for data vqs

2023-12-07 Thread Si-Wei Liu
the descriptor group for data vqs. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 89 1 file changed, 89 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 887c329..0cf3147 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c

[PATCH 24/40] vdpa: factor out vhost_vdpa_dma_batch_end

2023-12-07 Thread Si-Wei Liu
Refactoring only. No functional change. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 30 ++ 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index b0239b8

[PATCH 32/40] vdpa: factor out vhost_vdpa_net_get_nc_vdpa

2023-12-07 Thread Si-Wei Liu
Introduce new API. No functional change on existing API. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 1c1d61f..683619f 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost

[PATCH 08/40] vdpa: add back vhost_vdpa_net_first_nc_vdpa

2023-12-07 Thread Si-Wei Liu
Previous commits had it removed. Now adding it back because this function will be needed by next patches. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index dbfa192

[PATCH 27/40] vdpa: add asid to all dma_batch call sites

2023-12-07 Thread Si-Wei Liu
Will allow other callers to specifcy asid when calling the dma_batch API. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 18 +- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index e0137f0..d3f5721 100644

[PATCH 16/40] vdpa: indicate SVQ switching via flag

2023-12-07 Thread Si-Wei Liu
svq_switching indicates the case where SVQ mode change is on going. Positive (1) means switching from the normal passthrough mode to SVQ mode, and negative (-1) meaning switch SVQ back to the passthrough; zero (0) indicates that there's no SVQ mode switch taking place. Signed-off-by: Si-Wei Liu

[PATCH 37/40] vdpa: add vhost_vdpa_set_dev_vring_base trace for svq mode

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 5 - 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index a8d3321..5085607 100644 --- a/hw/virtio

[PATCH 36/40] vdpa: add vhost_vdpa_get_vring_base trace for svq mode

2023-12-07 Thread Si-Wei Liu
For better debuggability and observability. Signed-off-by: Si-Wei Liu --- hw/virtio/trace-events | 2 +- hw/virtio/vhost-vdpa.c | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 196f32f..a8d3321 100644 --- a/hw/virtio

[PATCH 01/40] linux-headers: add vhost_types.h and vhost.h

2023-12-07 Thread Si-Wei Liu
Signed-off-by: Si-Wei Liu --- include/standard-headers/linux/vhost_types.h | 13 + linux-headers/linux/vhost.h | 9 + 2 files changed, 22 insertions(+) diff --git a/include/standard-headers/linux/vhost_types.h b/include/standard-headers/linux/vhost_types.h

[PATCH 10/40] vdpa: assign svq descriptors a separate ASID when possible

2023-12-07 Thread Si-Wei Liu
address space ID to host SVQ descriptors because both CVQ and SVQ are emulated in the same QEMU process, which will share the same VA address space. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 5 - net/vhost-vdpa.c | 57 ++ 2 files

[PATCH 23/40] vdpa: vhost_vdpa_dma_batch_begin_once rename

2023-12-07 Thread Si-Wei Liu
No functional changes. Rename only. Signed-off-by: Si-Wei Liu --- hw/virtio/vhost-vdpa.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 7a1b7f4..a6c6fe5 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost

[PATCH 02/40] vdpa: add vhost_vdpa_get_vring_desc_group

2023-12-07 Thread Si-Wei Liu
Internal API to get the descriptor group index for a specific virtqueue through the VHOST_VDPA_GET_VRING_DESC_GROUP ioctl. Signed-off-by: Si-Wei Liu --- net/vhost-vdpa.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 90f4128

[PATCH 05/40] vdpa: populate desc_group from net_vhost_vdpa_init

2023-12-07 Thread Si-Wei Liu
will be set to a negative value -1. Signed-off-by: Si-Wei Liu --- include/hw/virtio/vhost-vdpa.h | 1 + net/vhost-vdpa.c | 15 +-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 6533ad2

[PATCH 00/40] vdpa-net: improve migration downtime through descriptor ASID and persistent IOTLB

2023-12-07 Thread Si-Wei Liu
.html [2] VHOST_BACKEND_F_DESC_ASID https://lore.kernel.org/virtualization/20231018171456.1624030-2-dtatu...@nvidia.com/ [3] VHOST_BACKEND_F_IOTLB_PERSIST https://lore.kernel.org/virtualization/1698304480-18463-1-git-send-email-si-wei@oracle.com/ --- Si-Wei Liu (40): linux-headers: add

Re: [PATCH 9.0 04/13] vdpa: move shadow_data to vhost_vdpa_shared

2023-12-05 Thread Si-Wei Liu
vhost_vdpa.shadow_vqs_enabled; -v->shadow_vqs_enabled = s0->vhost_vdpa.shadow_vqs_enabled; + v->shadow_vqs_enabled = v->shared->shadow_data; This new code looks fine. Reviewed-by: Si-Wei Liu s->vhost_vdpa.address_space_id = VHOST_VDPA_GUEST_PA_ASID; -if (

Re: [RFC PATCH 00/18] Map memory at destination .load_setup in vDPA-net migration

2023-12-05 Thread Si-Wei Liu
On 12/5/2023 6:23 AM, Eugenio Perez Martin wrote: On Fri, Nov 3, 2023 at 9:19 PM Si-Wei Liu wrote: On 11/2/2023 5:37 AM, Eugenio Perez Martin wrote: On Thu, Nov 2, 2023 at 11:13 AM Si-Wei Liu wrote: On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Current memory operations like pinning

Re: [RFC PATCH 00/18] Map memory at destination .load_setup in vDPA-net migration

2023-11-03 Thread Si-Wei Liu
On 11/2/2023 3:12 AM, Si-Wei Liu wrote: On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Current memory operations like pinning may take a lot of time at the destination.  Currently they are done after the source of the migration is stopped, and before the workload is resumed

Re: [RFC PATCH 00/18] Map memory at destination .load_setup in vDPA-net migration

2023-11-03 Thread Si-Wei Liu
On 11/2/2023 5:37 AM, Eugenio Perez Martin wrote: On Thu, Nov 2, 2023 at 11:13 AM Si-Wei Liu wrote: On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Current memory operations like pinning may take a lot of time at the destination. Currently they are done after the source of the migration

Re: [RFC PATCH 00/18] Map memory at destination .load_setup in vDPA-net migration

2023-11-02 Thread Si-Wei Liu
On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Current memory operations like pinning may take a lot of time at the destination. Currently they are done after the source of the migration is stopped, and before the workload is resumed at the destination. This is a period where neigher

Re: [RFC PATCH 02/18] vdpa: move iova tree to the shared struct

2023-11-02 Thread Si-Wei Liu
On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination

Re: [RFC PATCH 15/18] vdpa: add vhost_vdpa_load_setup

2023-11-02 Thread Si-Wei Liu
On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Callers can use this function to setup the incoming migration. Signed-off-by: Eugenio Pérez --- include/hw/virtio/vhost-vdpa.h | 7 +++ hw/virtio/vhost-vdpa.c | 17 - 2 files changed, 23 insertions(+), 1

Re: [RFC PATCH 04/18] vdpa: move shadow_data to vhost_vdpa_shared

2023-11-02 Thread Si-Wei Liu
On 10/19/2023 7:34 AM, Eugenio Pérez wrote: Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination

Re: [PATCH] vhost: Perform memory section dirty scans once per iteration

2023-10-17 Thread Si-Wei Liu
On 10/6/2023 2:48 AM, Michael S. Tsirkin wrote: On Fri, Oct 06, 2023 at 09:58:30AM +0100, Joao Martins wrote: On 03/10/2023 15:01, Michael S. Tsirkin wrote: On Wed, Sep 27, 2023 at 12:14:28PM +0100, Joao Martins wrote: On setups with one or more virtio-net devices with vhost on, dirty

Re: [PATCH v3 0/5] Enable vdpa net migration with features depending on CVQ

2023-09-15 Thread Si-Wei Liu
Does this series need to work with the recently merged ENABLE_AFTER_DRIVER_OK series from kernel? -Siwei On 8/22/2023 1:53 AM, Eugenio Pérez wrote: At this moment the migration of net features that depends on CVQ is not possible, as there is no reliable way to restore the device state like

  1   2   3   4   5   >