Re: [question] VFIO Device Migration: The vCPU may be paused during vfio device DMA in iommu nested stage mode && vSVA

2021-09-24 Thread Kirti Wankhede
On 9/24/2021 12:17 PM, Tian, Kevin wrote: From: Kunkun Jiang Sent: Friday, September 24, 2021 2:19 PM Hi all, I encountered a problem in vfio device migration test. The vCPU may be paused during vfio-pci DMA in iommu nested stage mode && vSVA. This may lead to migration fail and other

Re: [PATCH 07/16] vfio: Avoid error_propagate() after migrate_add_blocker()

2021-07-21 Thread Kirti Wankhede
agate() after migrate_add_blocker()". Cc: Kirti Wankhede Cc: Alex Williamson Signed-off-by: Markus Armbruster --- hw/vfio/migration.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 82f654afb6..ff6b45de6b 100644 ---

Re: [PATCH v2 17/21] contrib/gitdm: add domain-map for NVIDIA

2021-07-15 Thread Kirti Wankhede
On 7/14/2021 11:50 PM, Alex Bennée wrote: Signed-off-by: Alex Bennée Cc: Kirti Wankhede Cc: Yishai Hadas Message-Id: <20210714093638.21077-18-alex.ben...@linaro.org> --- contrib/gitdm/domain-map | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/gitdm/domain-map b/contrib

Re: [PATCH v1 1/1] vfio: Make migration support non experimental by default.

2021-07-14 Thread Kirti Wankhede
. Signed-off-by: Tarun Gupta Signed-off-by: Kirti Wankhede --- hw/vfio/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index f74be78209..15e26f460b 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3199,7 +3199,7 @@ static Property

Re: [PATCH v1 1/1] vfio/migration: Correct device state from vmstate change for savevm case.

2021-06-18 Thread Kirti Wankhede
CCing more Nvidia folks who are testing this patch. Gentle Ping for review. Thanks, Kirti On 6/9/2021 12:07 AM, Kirti Wankhede wrote: Set _SAVING flag for device state from vmstate change handler when it gets called from savevm. Currently State transition savevm/suspend is seen

[PATCH v1 1/1] vfio/migration: Correct device state from vmstate change for savevm case.

2021-06-08 Thread Kirti Wankhede
tate transition from _RUNNING to _STOP occurs from vfio_vmstate_change() where when vmstate changes from running to !running, _RUNNING flag is reset but at the same time when vfio_vmstate_change() is called for RUN_STATE_SAVE_VM, _SAVING bit should be set. Reported by: Yishai Hadas Signed-off-by:

Re: [PATCH] vfio: Fix unregister SaveVMHandler in vfio_migration_finalize

2021-05-28 Thread Kirti Wankhede
first and then call register_savevm_live(). I am wondering whether we need to add 'VMSTATEIf *obj' in register_savevm_live(). What do you think of this? I think proposed change above is independent of this fix. I'll defer to other experts. Reviewed by: Kirti Wankhede

Re: [PATCH v3 1/3] vfio: Move the saving of the config space to the right place in VFIO migration

2021-02-28 Thread Kirti Wankhede
Reviewed-by: Kirti Wankhede On 2/23/2021 7:52 AM, Shenming Lu wrote: On ARM64 the VFIO SET_IRQS ioctl is dependent on the VM interrupt setup, if the restoring of the VFIO PCI device config space is before the VGIC, an error might occur in the kernel. So we move the saving of the config

Re: [RFC PATCH v2 1/3] vfio: Move the saving of the config space to the right place in VFIO migration

2021-02-18 Thread Kirti Wankhede
On 1/27/2021 3:06 AM, Alex Williamson wrote: On Thu, 10 Dec 2020 10:21:21 +0800 Shenming Lu wrote: On 2020/12/10 2:34, Alex Williamson wrote: On Wed, 9 Dec 2020 13:29:47 +0100 Cornelia Huck wrote: On Wed, 9 Dec 2020 16:09:17 +0800 Shenming Lu wrote: On ARM64 the VFIO SET_IRQS

Re: [RFC PATCH v2 1/3] vfio: Move the saving of the config space to the right place in VFIO migration

2021-02-18 Thread Kirti Wankhede
On 12/9/2020 1:39 PM, Shenming Lu wrote: On ARM64 the VFIO SET_IRQS ioctl is dependent on the VM interrupt setup, if the restoring of the VFIO PCI device config space is before the VGIC, an error might occur in the kernel. So we move the saving of the config space to the non-iterable

Re: [RFC PATCH v2 1/3] vfio: Move the saving of the config space to the right place in VFIO migration

2021-01-27 Thread Kirti Wankhede
On 1/27/2021 3:06 AM, Alex Williamson wrote: On Thu, 10 Dec 2020 10:21:21 +0800 Shenming Lu wrote: On 2020/12/10 2:34, Alex Williamson wrote: On Wed, 9 Dec 2020 13:29:47 +0100 Cornelia Huck wrote: On Wed, 9 Dec 2020 16:09:17 +0800 Shenming Lu wrote: On ARM64 the VFIO SET_IRQS

Re: [PATCH] vfio/migrate: Move switch of dirty tracking into vfio_memory_listener

2021-01-27 Thread Kirti Wankhede
On 1/11/2021 1:04 PM, Keqian Zhu wrote: For now the switch of vfio dirty page tracking is integrated into the vfio_save_handler, it causes some problems [1]. Sorry, I missed [1] mail, somehow it didn't landed in my inbox. The object of dirty tracking is guest memory, but the object of

[PATCH v2 1/1] Fix to show vfio migration stat in migration status

2020-12-01 Thread Kirti Wankhede
stats") Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 12 +++- include/hw/vfio/vfio-common.h | 1 - include/hw/vfio/vfio.h| 2 ++ migration/migration.c | 16 +--- stubs/meson.build | 1 + stubs/vfio.c

Re: [PATCH 1/1] Fix to show vfio migration stat in migration status

2020-11-25 Thread Kirti Wankhede
On 11/26/2020 12:33 AM, Dr. David Alan Gilbert wrote: * Kirti Wankhede (kwankh...@nvidia.com) wrote: On 11/25/2020 3:00 PM, Dr. David Alan Gilbert wrote: * Kirti Wankhede (kwankh...@nvidia.com) wrote: Header file where CONFIG_VFIO is defined is not included in migration.c file. Include

Re: [PATCH 1/1] Fix to show vfio migration stat in migration status

2020-11-25 Thread Kirti Wankhede
On 11/25/2020 3:00 PM, Dr. David Alan Gilbert wrote: * Kirti Wankhede (kwankh...@nvidia.com) wrote: Header file where CONFIG_VFIO is defined is not included in migration.c file. Include config devices header file in migration.c. Fixes: 3710586caa5d ("qapi: Add VFIO devices migration

Re: [PATCH 1/1] Fix to show vfio migration stat in migration status

2020-11-23 Thread Kirti Wankhede
On 11/23/2020 10:03 PM, Alex Williamson wrote: On Thu, 19 Nov 2020 01:58:47 +0530 Kirti Wankhede wrote: Header file where CONFIG_VFIO is defined is not included in migration.c file. Include config devices header file in migration.c. Fixes: 3710586caa5d ("qapi: Add VFIO devices migr

[PATCH v2 1/1] vfio: Change default dirty pages tracking behavior during migration

2020-11-23 Thread Kirti Wankhede
tracking during iterative phase will be disabled. Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 11 +++ hw/vfio/pci.c | 3 +++ include/hw/vfio/vfio-common.h | 1 + 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio

Re: [v2 1/1] vfio: Change default dirty pages tracking behavior during migration

2020-11-23 Thread Kirti Wankhede
Sorry for spam, resending it again with 'PATCH'in subject. Kirti. On 11/23/2020 7:38 PM, Kirti Wankhede wrote: By default dirty pages tracking is enabled during iterative phase (pre-copy phase). Added per device opt-out option 'pre-copy-dirty-page-tracking' to disable dirty pages tracking

[v2 1/1] vfio: Change default dirty pages tracking behavior during migration

2020-11-23 Thread Kirti Wankhede
tracking during iterative phase will be disabled. Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 11 +++ hw/vfio/pci.c | 3 +++ include/hw/vfio/vfio-common.h | 1 + 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio

Re: [PATCH RFC] vfio: Move the saving of the config space to the right place in VFIO migration

2020-11-19 Thread Kirti Wankhede
On 11/14/2020 2:47 PM, Shenming Lu wrote: When running VFIO migration, I found that the restoring of VFIO PCI device’s config space is before VGIC on ARM64 target. But generally, interrupt controllers need to be restored before PCI devices. Is there any other way by which VGIC can be

[PATCH 1/1] vfio: Change default dirty pages tracking behavior during migration

2020-11-18 Thread Kirti Wankhede
tracking during iterative phase will be disabled. Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 11 +++ hw/vfio/pci.c | 3 +++ include/hw/vfio/vfio-common.h | 1 + 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio

Re: [PATCH RFC] vfio: Set the priority of VFIO VM state change handler explicitly

2020-11-18 Thread Kirti Wankhede
migration->migration_state.notify = vfio_migration_state_notifier; add_migration_state_change_notifier(>migration_state); Looks good to me. Reviewed-by: Kirti Wankhede

[PATCH 1/1] Fix to show vfio migration stat in migration status

2020-11-18 Thread Kirti Wankhede
Header file where CONFIG_VFIO is defined is not included in migration.c file. Include config devices header file in migration.c. Fixes: 3710586caa5d ("qapi: Add VFIO devices migration stats in Migration stats") Signed-off-by: Kirti Wankhede --- meson.build | 1 +

Re: [RFC PATCH for-QEMU-5.2] vfio: Make migration support experimental

2020-11-10 Thread Kirti Wankhede
wrong from the point of view of the user. Dave Alex Reviewed-by: Dr. David Alan Gilbert Link: https://lists.gnu.org/archive/html/qemu-devel/2020-11/msg00807.html Cc: Kirti Wankhede Cc: Neo Jia Cc: Dr. David Alan Gilbert Cc: Juan Quintela Cc: Philippe Mathieu-Daudé Cc: Cornelia Huck

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-11-06 Thread Kirti Wankhede
On 11/6/2020 2:56 AM, Alex Williamson wrote: On Fri, 6 Nov 2020 02:22:11 +0530 Kirti Wankhede wrote: On 11/6/2020 12:41 AM, Alex Williamson wrote: On Fri, 6 Nov 2020 00:29:36 +0530 Kirti Wankhede wrote: On 11/4/2020 6:15 PM, Alex Williamson wrote: On Wed, 4 Nov 2020 13:25:40 +0530

[PATCH v2 1/1] Fix use after free in vfio_migration_probe

2020-11-06 Thread Kirti Wankhede
Fixes Coverity issue: CID 1436126: Memory - illegal accesses (USE_AFTER_FREE) Fixes: a9e271ec9b36 ("vfio: Add migration region initialization and finalize function") Signed-off-by: Kirti Wankhede Reviewed-by: David Edmondson Reviewed-by: Alex Bennée Reviewed-by: Philippe Mat

[PATCH 1/1] Change the order of g_free(info) and tracepoint

2020-11-06 Thread Kirti Wankhede
Fixes Coverity issue: CID 1436126: Memory - illegal accesses (USE_AFTER_FREE) Fixes: a9e271ec9b36 ("vfio: Add migration region initialization and finalize function") Signed-off-by: Kirti Wankhede --- hw/vfio/migration.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-11-05 Thread Kirti Wankhede
On 11/6/2020 12:41 AM, Alex Williamson wrote: On Fri, 6 Nov 2020 00:29:36 +0530 Kirti Wankhede wrote: On 11/4/2020 6:15 PM, Alex Williamson wrote: On Wed, 4 Nov 2020 13:25:40 +0530 Kirti Wankhede wrote: On 11/4/2020 1:57 AM, Alex Williamson wrote: On Wed, 4 Nov 2020 01:18:12 +0530

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-11-05 Thread Kirti Wankhede
On 11/4/2020 6:15 PM, Alex Williamson wrote: On Wed, 4 Nov 2020 13:25:40 +0530 Kirti Wankhede wrote: On 11/4/2020 1:57 AM, Alex Williamson wrote: On Wed, 4 Nov 2020 01:18:12 +0530 Kirti Wankhede wrote: On 10/30/2020 12:35 AM, Alex Williamson wrote: On Thu, 29 Oct 2020 23:11:16

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-11-03 Thread Kirti Wankhede
On 11/4/2020 1:57 AM, Alex Williamson wrote: On Wed, 4 Nov 2020 01:18:12 +0530 Kirti Wankhede wrote: On 10/30/2020 12:35 AM, Alex Williamson wrote: On Thu, 29 Oct 2020 23:11:16 +0530 Kirti Wankhede wrote: +System memory dirty pages tracking

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-11-03 Thread Kirti Wankhede
On 10/30/2020 12:35 AM, Alex Williamson wrote: On Thu, 29 Oct 2020 23:11:16 +0530 Kirti Wankhede wrote: +System memory dirty pages tracking +-- + +A ``log_sync`` memory listener callback is added to mark system memory pages s/is added to mark/marks

Re: Out-of-Process Device Emulation session at KVM Forum 2020

2020-10-29 Thread Kirti Wankhede
On 10/29/2020 10:12 PM, Daniel P. Berrangé wrote: On Thu, Oct 29, 2020 at 04:15:30PM +, David Edmondson wrote: On Thursday, 2020-10-29 at 21:02:05 +08, Jason Wang wrote: 2) Did qemu even try to migrate opaque blobs before? It's probably a bad design of migration protocol as well. The

Re: [PATCH v1] docs/devel: Add VFIO device migration documentation

2020-10-29 Thread Kirti Wankhede
Thanks for corrections Cornelia. I had done the corrections you suggested I had not replied, see my comments on couple of places where I disagree. On 10/29/2020 5:22 PM, Cornelia Huck wrote: On Thu, 29 Oct 2020 11:23:11 +0530 Kirti Wankhede wrote: Document interfaces used for VFIO device

[PATCH v1] docs/devel: Add VFIO device migration documentation

2020-10-29 Thread Kirti Wankhede
Document interfaces used for VFIO device migration. Added flow of state changes during live migration with VFIO device. Signed-off-by: Kirti Wankhede --- MAINTAINERS | 1 + docs/devel/vfio-migration.rst | 119 ++ 2 files changed, 120

Re: [PATCH v29 05/17] vfio: Add VM state change handler to know state of VM

2020-10-26 Thread Kirti Wankhede
On 10/26/2020 7:32 PM, Alex Williamson wrote: On Mon, 26 Oct 2020 19:18:51 +0530 Kirti Wankhede wrote: On 10/26/2020 6:30 PM, Alex Williamson wrote: On Mon, 26 Oct 2020 15:06:15 +0530 Kirti Wankhede wrote: VM state change handler is called on change in VM's state. Based on VM state

Re: [PATCH v29 05/17] vfio: Add VM state change handler to know state of VM

2020-10-26 Thread Kirti Wankhede
On 10/26/2020 6:30 PM, Alex Williamson wrote: On Mon, 26 Oct 2020 15:06:15 +0530 Kirti Wankhede wrote: VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function

[PATCH v29 14/17] vfio: Dirty page tracking when vIOMMU is enabled

2020-10-26 Thread Kirti Wankhede
When vIOMMU is enabled, register MAP notifier from log_sync when all devices in container are in stop and copy phase of migration. Call replay and get dirty pages from notifier callback. Suggested-by: Alex Williamson Signed-off-by: Kirti Wankhede Reviewed-by: Yan Zhao --- hw/vfio/common.c

[PATCH v29 11/17] vfio: Get migration capability flags for container

2020-10-26 Thread Kirti Wankhede
if IOMMU module doesn't support migration capability. Signed-off-by: Kirti Wankhede Cc: Shameer Kolothum Cc: Eric Auger --- hw/vfio/common.c | 90 +++ hw/vfio/migration.c | 7 +++- include/hw/vfio/vfio-common.h | 3 ++ 3 files

[PATCH v29 17/17] qapi: Add VFIO devices migration stats in Migration stats

2020-10-26 Thread Kirti Wankhede
Added amount of bytes transferred to the VM at destination by all VFIO devices Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/common.c | 19 +++ hw/vfio/migration.c | 9 + include/hw/vfio/vfio-common.h | 3

[PATCH v29 09/17] vfio: Add load state functions to SaveVMHandlers

2020-10-26 Thread Kirti Wankhede
is written in staging buffer. For user, data is opaque. User should write data in the same order as received. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Yan Zhao --- hw/vfio/migration.c | 195

[PATCH v29 16/17] vfio: Make vfio-pci device migration capable

2020-10-26 Thread Kirti Wankhede
from generic structure of VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Cornelia Huck --- hw/vfio/pci.c | 28 hw/vfio/pci.h | 1 - 2 files changed, 8 insertions(+), 21 deletions(-) diff --git a/hw

[PATCH v29 06/17] vfio: Add migration state change notifier

2020-10-26 Thread Kirti Wankhede
Added migration state change notifier to get notification on migration state change. These states are translated to VFIO device state and conveyed to vendor driver. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Cornelia Huck --- hw/vfio

[PATCH v29 10/17] memory: Set DIRTY_MEMORY_MIGRATION when IOMMU is enabled

2020-10-26 Thread Kirti Wankhede
mr->ram_block is NULL when mr->is_iommu is true, then fr.dirty_log_mask wasn't set correctly due to which memory listener's log_sync doesn't get called. This patch returns log_mask with DIRTY_MEMORY_MIGRATION set when IOMMU is enabled. Signed-off-by: Kirti Wankhede Reviewed-by: Ya

[PATCH v29 13/17] vfio: Add vfio_listener_log_sync to mark dirty pages

2020-10-26 Thread Kirti Wankhede
vfio_listener_log_sync gets list of dirty pages from container using VFIO_IOMMU_GET_DIRTY_BITMAP ioctl and mark those pages dirty when all devices are stopped and saving state. Return early for the RAM block section of mapped MMIO region. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia

[PATCH v29 07/17] vfio: Register SaveVMHandlers for VFIO device

2020-10-26 Thread Kirti Wankhede
, VM is running when .save_setup is called, _SAVING | _RUNNING state is set for VFIO device. During save-restore, VM is paused, _SAVING state is set for VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Cornelia Huck Reviewed-by: Yan Zhao --- hw/vfio/migration.c |

[PATCH v29 12/17] vfio: Add function to start and stop dirty pages tracking

2020-10-26 Thread Kirti Wankhede
Call VFIO_IOMMU_DIRTY_PAGES ioctl to start and stop dirty pages tracking for VFIO devices. Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 36 1 file changed, 36 insertions(+) diff --git a/hw/vfio/migration.c b

[PATCH v29 15/17] vfio: Add ioctl to get dirty pages bitmap during dma unmap

2020-10-26 Thread Kirti Wankhede
-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/common.c | 96 +--- 1 file changed, 92 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index c0b5b6245a47..49c68a5253ae 100644 --- a/hw/vfio/common.c +++ b/hw/vfio

[PATCH v29 08/17] vfio: Add save state functions to SaveVMHandlers

2020-10-26 Thread Kirti Wankhede
Zhi Wang Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Yan Zhao --- hw/vfio/migration.c | 276 ++ hw/vfio/trace-events | 6 + include/hw/vfio/vfio-common.h | 1 + 3 files changed, 283 insertions(+) diff --git

[PATCH v29 05/17] vfio: Add VM state change handler to know state of VM

2020-10-26 Thread Kirti Wankhede
VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function to set device_state. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert

[PATCH v29 04/17] vfio: Add migration region initialization and finalize function

2020-10-26 Thread Kirti Wankhede
Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful then migration is supported else migration is blocked. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Acked

[PATCH v29 02/17] vfio: Add vfio_get_object callback to VFIODeviceOps

2020-10-26 Thread Kirti Wankhede
Hook vfio_get_object callback for PCI devices. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Suggested-by: Cornelia Huck Reviewed-by: Cornelia Huck --- hw/vfio/pci.c | 8 include/hw/vfio/vfio-common.h | 1 + 2 files changed, 9 insertions(+) diff --git a/hw/vfio

[PATCH v29 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-10-26 Thread Kirti Wankhede
Added functions to save and restore PCI device specific data, specifically config space of PCI device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/pci.c | 51 +++ include/hw/vfio/vfio-common.h | 2 ++ 2 files changed

[PATCH v29 00/17] Add migration support for VFIO devices

2020-10-26 Thread Kirti Wankhede
tion region. - Added both type of access support, trapped or mmapped, for data section of the region. - Moved PCI device functions to pci file. - Added iteration to get dirty page bitmap until bitmap for all requested pages are copied. Thanks, Kirti Kirti Wankhede (17): vfio: Add function

[PATCH v29 01/17] vfio: Add function to unmap VFIO region

2020-10-26 Thread Kirti Wankhede
This function will be used for migration region. Migration region is mmaped when migration starts and will be unmapped when migration is complete. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Cornelia Huck --- hw/vfio/common.c | 32

Re: [PATCH v28 07/17] vfio: Register SaveVMHandlers for VFIO device

2020-10-24 Thread Kirti Wankhede
On 10/24/2020 4:56 PM, Yan Zhao wrote: On Fri, Oct 23, 2020 at 04:10:33PM +0530, Kirti Wankhede wrote: Define flags to be used as delimiter in migration stream for VFIO devices. Added .save_setup and .save_cleanup functions. Map & unmap migration region from these functions at source du

Re: [PATCH v28 05/17] vfio: Add VM state change handler to know state of VM

2020-10-24 Thread Kirti Wankhede
On 10/23/2020 5:02 PM, Cornelia Huck wrote: On Fri, 23 Oct 2020 16:10:31 +0530 Kirti Wankhede wrote: VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function

Re: [PATCH v28 04/17] vfio: Add migration region initialization and finalize function

2020-10-24 Thread Kirti Wankhede
On 10/23/2020 4:54 PM, Cornelia Huck wrote: On Fri, 23 Oct 2020 16:10:30 +0530 Kirti Wankhede wrote: Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful

Re: [PATCH v28 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-10-24 Thread Kirti Wankhede
fig space value using pci_get_word() - both are 0x507. VM restores successfully. Yan, can you share pci command values before and after as above? what exactly is missing? Thanks, Kirti Thanks, Alex On Fri, Oct 23, 2020 at 04:10:29PM +0530, Kirti Wankhede wrote: Added functions to save a

Re: [PATCH v28 00/17] Add migration support for VFIO devices

2020-10-24 Thread Kirti Wankhede
On 10/24/2020 10:26 PM, Philippe Mathieu-Daudé wrote: Hi Kirti, On 10/23/20 12:40 PM, Kirti Wankhede wrote: Hi, This Patch set adds migration support for VFIO devices in QEMU. ... Since there is no device which has hardware support for system memmory dirty bitmap tracking, right now

Re: [PATCH v28 04/17] vfio: Add migration region initialization and finalize function

2020-10-24 Thread Kirti Wankhede
On 10/24/2020 7:51 PM, Alex Williamson wrote: On Sat, 24 Oct 2020 15:09:14 +0530 Kirti Wankhede wrote: On 10/23/2020 10:22 PM, Alex Williamson wrote: On Fri, 23 Oct 2020 16:10:30 +0530 Kirti Wankhede wrote: Whether the VFIO device supports migration or not is decided based

Re: [PATCH v28 04/17] vfio: Add migration region initialization and finalize function

2020-10-24 Thread Kirti Wankhede
On 10/23/2020 10:22 PM, Alex Williamson wrote: On Fri, 23 Oct 2020 16:10:30 +0530 Kirti Wankhede wrote: Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful

[PATCH v28 17/17] qapi: Add VFIO devices migration stats in Migration stats

2020-10-23 Thread Kirti Wankhede
Added amount of bytes transferred to the VM at destination by all VFIO devices Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/common.c | 19 +++ hw/vfio/migration.c | 9 + include/hw/vfio/vfio-common.h | 3

[PATCH v28 12/17] vfio: Add function to start and stop dirty pages tracking

2020-10-23 Thread Kirti Wankhede
Call VFIO_IOMMU_DIRTY_PAGES ioctl to start and stop dirty pages tracking for VFIO devices. Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 36 1 file changed, 36 insertions(+) diff --git a/hw/vfio/migration.c b

[PATCH v28 16/17] vfio: Make vfio-pci device migration capable

2020-10-23 Thread Kirti Wankhede
from generic structure of VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Cornelia Huck --- hw/vfio/pci.c | 28 hw/vfio/pci.h | 1 - 2 files changed, 8 insertions(+), 21 deletions(-) diff --git a/hw

[PATCH v28 15/17] vfio: Add ioctl to get dirty pages bitmap during dma unmap

2020-10-23 Thread Kirti Wankhede
-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/common.c | 96 +--- 1 file changed, 92 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index c0b5b6245a47..49c68a5253ae 100644 --- a/hw/vfio/common.c +++ b/hw/vfio

[PATCH v28 11/17] vfio: Get migration capability flags for container

2020-10-23 Thread Kirti Wankhede
if IOMMU module doesn't support migration capability. Signed-off-by: Kirti Wankhede Cc: Shameer Kolothum Cc: Eric Auger --- hw/vfio/common.c | 90 +++ hw/vfio/migration.c | 7 +++- include/hw/vfio/vfio-common.h | 3 ++ 3 files

[PATCH v28 14/17] vfio: Dirty page tracking when vIOMMU is enabled

2020-10-23 Thread Kirti Wankhede
When vIOMMU is enabled, add MAP notifier from log_sync when all devices in container are in stop and copy phase of migration. Call replay and then from notifier callback, get dirty pages. Suggested-by: Alex Williamson Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 88

[PATCH v28 08/17] vfio: Add save state functions to SaveVMHandlers

2020-10-23 Thread Kirti Wankhede
Zhi Wang Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/migration.c | 276 ++ hw/vfio/trace-events | 6 + include/hw/vfio/vfio-common.h | 1 + 3 files changed, 283 insertions(+) diff --git a/hw/vfio/migration.c

[PATCH v28 06/17] vfio: Add migration state change notifier

2020-10-23 Thread Kirti Wankhede
Added migration state change notifier to get notification on migration state change. These states are translated to VFIO device state and conveyed to vendor driver. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 28

[PATCH v28 07/17] vfio: Register SaveVMHandlers for VFIO device

2020-10-23 Thread Kirti Wankhede
, VM is running when .save_setup is called, _SAVING | _RUNNING state is set for VFIO device. During save-restore, VM is paused, _SAVING state is set for VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/migration.c | 102 +++ hw/

[PATCH v28 09/17] vfio: Add load state functions to SaveVMHandlers

2020-10-23 Thread Kirti Wankhede
is written in staging buffer. For user, data is opaque. User should write data in the same order as received. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 195 +++ hw/vfio/trace

[PATCH v28 04/17] vfio: Add migration region initialization and finalize function

2020-10-23 Thread Kirti Wankhede
Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful then migration is supported else migration is blocked. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Acked

[PATCH v28 13/17] vfio: Add vfio_listener_log_sync to mark dirty pages

2020-10-23 Thread Kirti Wankhede
vfio_listener_log_sync gets list of dirty pages from container using VFIO_IOMMU_GET_DIRTY_BITMAP ioctl and mark those pages dirty when all devices are stopped and saving state. Return early for the RAM block section of mapped MMIO region. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia

[PATCH v28 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-10-23 Thread Kirti Wankhede
Added functions to save and restore PCI device specific data, specifically config space of PCI device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/pci.c | 48 +++ include/hw/vfio/vfio-common.h | 2 ++ 2 files changed

[PATCH v28 01/17] vfio: Add function to unmap VFIO region

2020-10-23 Thread Kirti Wankhede
This function will be used for migration region. Migration region is mmaped when migration starts and will be unmapped when migration is complete. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Cornelia Huck --- hw/vfio/common.c | 32

[PATCH v28 10/17] memory: Set DIRTY_MEMORY_MIGRATION when IOMMU is enabled

2020-10-23 Thread Kirti Wankhede
mr->ram_block is NULL when mr->is_iommu is true, then fr.dirty_log_mask wasn't set correctly due to which memory listener's log_sync doesn't get called. This patch returns log_mask with DIRTY_MEMORY_MIGRATION set when IOMMU is enabled. Signed-off-by: Kirti Wankhede --- softmmu/memory

[PATCH v28 05/17] vfio: Add VM state change handler to know state of VM

2020-10-23 Thread Kirti Wankhede
VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function to set device_state. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert

[PATCH v28 00/17] Add migration support for VFIO devices

2020-10-23 Thread Kirti Wankhede
device functions to pci file. - Added iteration to get dirty page bitmap until bitmap for all requested pages are copied. Thanks, Kirti Kirti Wankhede (17): vfio: Add function to unmap VFIO region vfio: Add vfio_get_object callback to VFIODeviceOps vfio: Add save and load funct

[PATCH v28 02/17] vfio: Add vfio_get_object callback to VFIODeviceOps

2020-10-23 Thread Kirti Wankhede
Hook vfio_get_object callback for PCI devices. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Suggested-by: Cornelia Huck Reviewed-by: Cornelia Huck --- hw/vfio/pci.c | 8 include/hw/vfio/vfio-common.h | 1 + 2 files changed, 9 insertions(+) diff --git a/hw/vfio

Re: [PATCH v27 17/17] qapi: Add VFIO devices migration stats in Migration stats

2020-10-23 Thread Kirti Wankhede
On 10/23/2020 3:48 AM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:42:07 +0530 Kirti Wankhede wrote: Added amount of bytes transferred to the VM at destination by all VFIO devices Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/common.c| 20

Re: [PATCH v27 09/17] vfio: Add load state functions to SaveVMHandlers

2020-10-23 Thread Kirti Wankhede
On 10/23/2020 1:20 AM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:41:59 +0530 Kirti Wankhede wrote: Sequence during _RESUMING device state: While data for this device is available, repeat below steps: a. read data_offset from where user application should write data. b. write data

Re: [PATCH v27 14/17] vfio: Dirty page tracking when vIOMMU is enabled

2020-10-23 Thread Kirti Wankhede
On 10/23/2020 2:07 AM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:42:04 +0530 Kirti Wankhede wrote: When vIOMMU is enabled, register MAP notifier from log_sync when all devices in container are in stop and copy phase of migration. Call replay and get dirty pages from notifier callback

Re: [PATCH v27 07/17] vfio: Register SaveVMHandlers for VFIO device

2020-10-23 Thread Kirti Wankhede
On 10/23/2020 12:21 AM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:41:57 +0530 Kirti Wankhede wrote: Define flags to be used as delimiter in migration stream for VFIO devices. Added .save_setup and .save_cleanup functions. Map & unmap migration region from these functions at so

Re: [PATCH v27 05/17] vfio: Add VM state change handler to know state of VM

2020-10-22 Thread Kirti Wankhede
On 10/22/2020 10:05 PM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:41:55 +0530 Kirti Wankhede wrote: VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function

Re: [PATCH v27 04/17] vfio: Add migration region initialization and finalize function

2020-10-22 Thread Kirti Wankhede
On 10/22/2020 7:52 PM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:41:54 +0530 Kirti Wankhede wrote: Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful

Re: [PATCH v27 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-10-22 Thread Kirti Wankhede
On 10/22/2020 7:36 PM, Alex Williamson wrote: On Thu, 22 Oct 2020 16:41:53 +0530 Kirti Wankhede wrote: Added functions to save and restore PCI device specific data, specifically config space of PCI device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/pci.c

Re: [PATCH v26 05/17] vfio: Add VM state change handler to know state of VM

2020-10-22 Thread Kirti Wankhede
On 10/22/2020 1:21 PM, Cornelia Huck wrote: On Wed, 21 Oct 2020 11:03:23 +0530 Kirti Wankhede wrote: On 10/20/2020 4:21 PM, Cornelia Huck wrote: On Sun, 18 Oct 2020 01:54:56 +0530 Kirti Wankhede wrote: On 9/29/2020 4:33 PM, Dr. David Alan Gilbert wrote: * Cornelia Huck (coh

[PATCH v27 17/17] qapi: Add VFIO devices migration stats in Migration stats

2020-10-22 Thread Kirti Wankhede
Added amount of bytes transferred to the VM at destination by all VFIO devices Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/common.c| 20 hw/vfio/migration.c | 10 ++ include/qemu/vfio-helpers.h | 3

[PATCH v27 14/17] vfio: Dirty page tracking when vIOMMU is enabled

2020-10-22 Thread Kirti Wankhede
When vIOMMU is enabled, register MAP notifier from log_sync when all devices in container are in stop and copy phase of migration. Call replay and get dirty pages from notifier callback. Suggested-by: Alex Williamson Signed-off-by: Kirti Wankhede --- hw/vfio/common.c | 95

[PATCH v27 05/17] vfio: Add VM state change handler to know state of VM

2020-10-22 Thread Kirti Wankhede
VM state change handler is called on change in VM's state. Based on VM state, VFIO device state should be changed. Added read/write helper functions for migration region. Added function to set device_state. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert

[PATCH v27 15/17] vfio: Add ioctl to get dirty pages bitmap during dma unmap

2020-10-22 Thread Kirti Wankhede
-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/common.c | 96 +--- 1 file changed, 92 insertions(+), 4 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 98c2b1f9b190..9c879e5c0f62 100644 --- a/hw/vfio/common.c +++ b/hw/vfio

[PATCH v27 13/17] vfio: Add vfio_listener_log_sync to mark dirty pages

2020-10-22 Thread Kirti Wankhede
vfio_listener_log_sync gets list of dirty pages from container using VFIO_IOMMU_GET_DIRTY_BITMAP ioctl and mark those pages dirty when all devices are stopped and saving state. Return early for the RAM block section of mapped MMIO region. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia

[PATCH v27 10/17] memory: Set DIRTY_MEMORY_MIGRATION when IOMMU is enabled

2020-10-22 Thread Kirti Wankhede
mr->ram_block is NULL when mr->is_iommu is true, then fr.dirty_log_mask wasn't set correctly due to which memory listener's log_sync doesn't get called. This patch returns log_mask with DIRTY_MEMORY_MIGRATION set when IOMMU is enabled. Signed-off-by: Kirti Wankhede --- softmmu/memory

[PATCH v27 07/17] vfio: Register SaveVMHandlers for VFIO device

2020-10-22 Thread Kirti Wankhede
, VM is running when .save_setup is called, _SAVING | _RUNNING state is set for VFIO device. During save-restore, VM is paused, _SAVING state is set for VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/migration.c | 96 hw/

[PATCH v27 16/17] vfio: Make vfio-pci device migration capable

2020-10-22 Thread Kirti Wankhede
from generic structure of VFIO device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert Reviewed-by: Cornelia Huck --- hw/vfio/pci.c | 28 hw/vfio/pci.h | 1 - 2 files changed, 8 insertions(+), 21 deletions(-) diff --git a/hw

[PATCH v27 12/17] vfio: Add function to start and stop dirty pages tracking

2020-10-22 Thread Kirti Wankhede
Call VFIO_IOMMU_DIRTY_PAGES ioctl to start and stop dirty pages tracking for VFIO devices. Signed-off-by: Kirti Wankhede Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 36 1 file changed, 36 insertions(+) diff --git a/hw/vfio/migration.c b

[PATCH v27 11/17] vfio: Get migration capability flags for container

2020-10-22 Thread Kirti Wankhede
if IOMMU module doesn't support migration capability. Signed-off-by: Kirti Wankhede Cc: Shameer Kolothum Cc: Eric Auger --- hw/vfio/common.c | 90 +++ hw/vfio/migration.c | 7 +++- include/hw/vfio/vfio-common.h | 3 ++ 3 files

[PATCH v27 03/17] vfio: Add save and load functions for VFIO PCI devices

2020-10-22 Thread Kirti Wankhede
Added functions to save and restore PCI device specific data, specifically config space of PCI device. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia --- hw/vfio/pci.c | 48 +++ include/hw/vfio/vfio-common.h | 2 ++ 2 files changed

[PATCH v27 09/17] vfio: Add load state functions to SaveVMHandlers

2020-10-22 Thread Kirti Wankhede
is written in staging buffer. For user, data is opaque. User should write data in the same order as received. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Reviewed-by: Dr. David Alan Gilbert --- hw/vfio/migration.c | 192 +++ hw/vfio/trace

[PATCH v27 04/17] vfio: Add migration region initialization and finalize function

2020-10-22 Thread Kirti Wankhede
Whether the VFIO device supports migration or not is decided based of migration region query. If migration region query is successful and migration region initialization is successful then migration is supported else migration is blocked. Signed-off-by: Kirti Wankhede Reviewed-by: Neo Jia Acked

  1   2   3   4   5   6   7   8   9   10   >