Re: [PATCH v15 Kernel 1/7] vfio: KABI for migration interface for device state
On 3/26/2020 3:03 PM, Christoph Hellwig wrote: s/KABI/UAPI/ in the subject and anywhere else in the series. Ok. Please avoid __packed__ structures and just properly pad them, they have a major performance impact on some platforms and will cause compiler warnings when taking addresses of members. Yes, removing it. Thanks, Kirti
Re: [PATCH v15 Kernel 1/7] vfio: KABI for migration interface for device state
s/KABI/UAPI/ in the subject and anywhere else in the series. Please avoid __packed__ structures and just properly pad them, they have a major performance impact on some platforms and will cause compiler warnings when taking addresses of members.
Re: [PATCH v15 Kernel 1/7] vfio: KABI for migration interface for device state
On 3/24/2020 2:00 AM, Auger Eric wrote:
Hi Kirti,
On 3/19/20 9:16 PM, Kirti Wankhede wrote:
- Defined MIGRATION region type and sub-type.
- Defined vfio_device_migration_info structure which will be placed at the
0th offset of migration region to get/set VFIO device related
information. Defined members of structure and usage on read/write access.
- Defined device states and state transition details.
- Defined sequence to be followed while saving and resuming VFIO device.
Signed-off-by: Kirti Wankhede
Reviewed-by: Neo Jia
Please forgive me, I have just discovered v15 was available.
hereafter, you will find the 2 main points I feel difficult to
understand when reading the documentation.
---
include/uapi/linux/vfio.h | 227 ++
1 file changed, 227 insertions(+)
diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
index 9e843a147ead..d0021467af53 100644
--- a/include/uapi/linux/vfio.h
+++ b/include/uapi/linux/vfio.h
@@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
#define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
#define VFIO_REGION_TYPE_GFX(1)
#define VFIO_REGION_TYPE_CCW (2)
+#define VFIO_REGION_TYPE_MIGRATION (3)
/* sub-types for VFIO_REGION_TYPE_PCI_* */
@@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
/* sub-types for VFIO_REGION_TYPE_CCW */
#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD (1)
+/* sub-types for VFIO_REGION_TYPE_MIGRATION */
+#define VFIO_REGION_SUBTYPE_MIGRATION (1)
+
+/*
+ * The structure vfio_device_migration_info is placed at the 0th offset of
+ * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device related
+ * migration information. Field accesses from this structure are only supported
+ * at their native width and alignment. Otherwise, the result is undefined and
+ * vendor drivers should return an error.
+ *
+ * device_state: (read/write)
+ * - The user application writes to this field to inform the vendor driver
+ *about the device state to be transitioned to.
+ * - The vendor driver should take the necessary actions to change the
+ *device state. After successful transition to a given state, the
+ *vendor driver should return success on write(device_state, state)
+ *system call. If the device state transition fails, the vendor driver
+ *should return an appropriate -errno for the fault condition.
+ * - On the user application side, if the device state transition fails,
+ * that is, if write(device_state, state) returns an error, read
+ * device_state again to determine the current state of the device from
+ * the vendor driver.
+ * - The vendor driver should return previous state of the device unless
+ *the vendor driver has encountered an internal error, in which case
+ *the vendor driver may report the device_state
VFIO_DEVICE_STATE_ERROR.
+ * - The user application must use the device reset ioctl to recover the
+ *device from VFIO_DEVICE_STATE_ERROR state. If the device is
+ *indicated to be in a valid device state by reading device_state, the
+ *user application may attempt to transition the device to any valid
+ *state reachable from the current state or terminate itself.
+ *
+ * device_state consists of 3 bits:
+ * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is clear,
+ *it indicates the _STOP state. When the device state is changed to
+ *_STOP, driver should stop the device before write() returns.
+ * - If bit 1 is set, it indicates the _SAVING state, which means that the
+ *driver should start gathering device state information that will be
+ *provided to the VFIO user application to save the device's state.
+ * - If bit 2 is set, it indicates the _RESUMING state, which means that
+ *the driver should prepare to resume the device. Data provided through
+ *the migration region should be used to resume the device.
+ * Bits 3 - 31 are reserved for future use. To preserve them, the user
+ * application should perform a read-modify-write operation on this
+ * field when modifying the specified bits.
+ *
+ * +--- _RESUMING
+ * |+-- _SAVING
+ * ||+- _RUNNING
+ * |||
+ * 000b => Device Stopped, not saving or resuming
+ * 001b => Device running, which is the default state
+ * 010b => Stop the device & save the device state, stop-and-copy state
+ * 011b => Device running and save the device state, pre-copy state
+ * 100b => Device stopped and the device state is resuming
+ * 101b => Invalid state
+ * 110b => Error state
+ * 111b => Invalid state
+ *
+ * State transitions:
+ *
+ * _RESUMING _RUNNINGPre-copyStop-and-copy _STOP
+ *(100b) (001b) (011b)(010b) (000b)
+ * 0. Running or
Re: [PATCH v15 Kernel 1/7] vfio: KABI for migration interface for device state
Hi Kirti,
On 3/19/20 9:16 PM, Kirti Wankhede wrote:
> - Defined MIGRATION region type and sub-type.
>
> - Defined vfio_device_migration_info structure which will be placed at the
> 0th offset of migration region to get/set VFIO device related
> information. Defined members of structure and usage on read/write access.
>
> - Defined device states and state transition details.
>
> - Defined sequence to be followed while saving and resuming VFIO device.
>
> Signed-off-by: Kirti Wankhede
> Reviewed-by: Neo Jia
Please forgive me, I have just discovered v15 was available.
hereafter, you will find the 2 main points I feel difficult to
understand when reading the documentation.
> ---
> include/uapi/linux/vfio.h | 227
> ++
> 1 file changed, 227 insertions(+)
>
> diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> index 9e843a147ead..d0021467af53 100644
> --- a/include/uapi/linux/vfio.h
> +++ b/include/uapi/linux/vfio.h
> @@ -305,6 +305,7 @@ struct vfio_region_info_cap_type {
> #define VFIO_REGION_TYPE_PCI_VENDOR_MASK (0x)
> #define VFIO_REGION_TYPE_GFX(1)
> #define VFIO_REGION_TYPE_CCW (2)
> +#define VFIO_REGION_TYPE_MIGRATION (3)
>
> /* sub-types for VFIO_REGION_TYPE_PCI_* */
>
> @@ -379,6 +380,232 @@ struct vfio_region_gfx_edid {
> /* sub-types for VFIO_REGION_TYPE_CCW */
> #define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD(1)
>
> +/* sub-types for VFIO_REGION_TYPE_MIGRATION */
> +#define VFIO_REGION_SUBTYPE_MIGRATION (1)
> +
> +/*
> + * The structure vfio_device_migration_info is placed at the 0th offset of
> + * the VFIO_REGION_SUBTYPE_MIGRATION region to get and set VFIO device
> related
> + * migration information. Field accesses from this structure are only
> supported
> + * at their native width and alignment. Otherwise, the result is undefined
> and
> + * vendor drivers should return an error.
> + *
> + * device_state: (read/write)
> + * - The user application writes to this field to inform the vendor
> driver
> + *about the device state to be transitioned to.
> + * - The vendor driver should take the necessary actions to change the
> + *device state. After successful transition to a given state, the
> + *vendor driver should return success on write(device_state, state)
> + *system call. If the device state transition fails, the vendor
> driver
> + *should return an appropriate -errno for the fault condition.
> + * - On the user application side, if the device state transition fails,
> + * that is, if write(device_state, state) returns an error, read
> + * device_state again to determine the current state of the device from
> + * the vendor driver.
> + * - The vendor driver should return previous state of the device unless
> + *the vendor driver has encountered an internal error, in which case
> + *the vendor driver may report the device_state
> VFIO_DEVICE_STATE_ERROR.
> + * - The user application must use the device reset ioctl to recover the
> + *device from VFIO_DEVICE_STATE_ERROR state. If the device is
> + *indicated to be in a valid device state by reading device_state,
> the
> + *user application may attempt to transition the device to any valid
> + *state reachable from the current state or terminate itself.
> + *
> + * device_state consists of 3 bits:
> + * - If bit 0 is set, it indicates the _RUNNING state. If bit 0 is
> clear,
> + *it indicates the _STOP state. When the device state is changed to
> + *_STOP, driver should stop the device before write() returns.
> + * - If bit 1 is set, it indicates the _SAVING state, which means that
> the
> + *driver should start gathering device state information that will be
> + *provided to the VFIO user application to save the device's state.
> + * - If bit 2 is set, it indicates the _RESUMING state, which means that
> + *the driver should prepare to resume the device. Data provided
> through
> + *the migration region should be used to resume the device.
> + * Bits 3 - 31 are reserved for future use. To preserve them, the user
> + * application should perform a read-modify-write operation on this
> + * field when modifying the specified bits.
> + *
> + * +--- _RESUMING
> + * |+-- _SAVING
> + * ||+- _RUNNING
> + * |||
> + * 000b => Device Stopped, not saving or resuming
> + * 001b => Device running, which is the default state
> + * 010b => Stop the device & save the device state, stop-and-copy state
> + * 011b => Device running and save the device state, pre-copy state
> + * 100b => Device stopped and the device state is resuming
> + * 101b => Invalid state
> + * 110b => Error state
> + * 111b => Invalid state
> + *
> + * State transitions:
> + *
> + *
