Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management

2021-01-12 Thread Davidlohr Bueso

On Tue, 12 Jan 2021, Shuo A Liu wrote:


On Mon 11.Jan'21 at 13:52:19 -0800, Davidlohr Bueso wrote:

Could this not be done in process context instead?


It could be. The original consideration with tasklet was more about
performance as the I/O requests dispatching is a hot code path. I think
irq thread has little performance impact? I can have a try to convert
the tasklet to irq thread.


Yes, there is some added latency between when the work is scheduled and
actually executed - however this should not be a problem for this scenario,
and furthermore consider that tasklets do not guarantee performance as
ksoftirqd comes in the picture under heavy load.

Thanks,
Davidlohr


Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management

2021-01-11 Thread Shuo A Liu

On Mon 11.Jan'21 at 13:52:19 -0800, Davidlohr Bueso wrote:

On Wed, 06 Jan 2021, shuo.a@intel.com wrote:

The processing flow of I/O requests are listed as following:

a) The I/O handler of the hypervisor will fill an I/O request with
 PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to
 the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to
 different registered clients based on the address of the I/O accesses,
 updates their state to PROCESSING, and notifies the corresponding
 client to handle.


Hmm so tasklets are deprecated (and have been for a while) and it's sad
to see incoming new users in modern Linux. This wouldn't be the first one,
however. We should be _removing_ users, not adding... In addition, this
expands the whole tasklet_disable/enable() hacks.


Sorry, i had not noticed that.



Could this not be done in process context instead?


It could be. The original consideration with tasklet was more about
performance as the I/O requests dispatching is a hot code path. I think
irq thread has little performance impact? I can have a try to convert
the tasklet to irq thread.

Thanks
shuo


Re: [PATCH v7 09/18] virt: acrn: Introduce I/O request management

2021-01-11 Thread Davidlohr Bueso

On Wed, 06 Jan 2021, shuo.a@intel.com wrote:

The processing flow of I/O requests are listed as following:

a) The I/O handler of the hypervisor will fill an I/O request with
  PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to
  the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to
  different registered clients based on the address of the I/O accesses,
  updates their state to PROCESSING, and notifies the corresponding
  client to handle.


Hmm so tasklets are deprecated (and have been for a while) and it's sad
to see incoming new users in modern Linux. This wouldn't be the first one,
however. We should be _removing_ users, not adding... In addition, this
expands the whole tasklet_disable/enable() hacks.

Could this not be done in process context instead?

Thanks,
Davidlohr


[PATCH v7 09/18] virt: acrn: Introduce I/O request management

2021-01-05 Thread shuo . a . liu
From: Shuo Liu 

An I/O request of a User VM, which is constructed by the hypervisor, is
distributed by the ACRN Hypervisor Service Module to an I/O client
corresponding to the address range of the I/O request.

For each User VM, there is a shared 4-KByte memory region used for I/O
requests communication between the hypervisor and Service VM. An I/O
request is a 256-byte structure buffer, which is 'struct
acrn_io_request', that is filled by an I/O handler of the hypervisor
when a trapped I/O access happens in a User VM. ACRN userspace in the
Service VM first allocates a 4-KByte page and passes the GPA (Guest
Physical Address) of the buffer to the hypervisor. The buffer is used as
an array of 16 I/O request slots with each I/O request slot being 256
bytes. This array is indexed by vCPU ID.

An I/O client, which is 'struct acrn_ioreq_client', is responsible for
handling User VM I/O requests whose accessed GPA falls in a certain
range. Multiple I/O clients can be associated with each User VM. There
is a special client associated with each User VM, called the default
client, that handles all I/O requests that do not fit into the range of
any other I/O clients. The ACRN userspace acts as the default client for
each User VM.

The state transitions of a ACRN I/O request are as follows.

   FREE -> PENDING -> PROCESSING -> COMPLETE -> FREE -> ...

FREE: this I/O request slot is empty
PENDING: a valid I/O request is pending in this slot
PROCESSING: the I/O request is being processed
COMPLETE: the I/O request has been processed

An I/O request in COMPLETE or FREE state is owned by the hypervisor. HSM
and ACRN userspace are in charge of processing the others.

The processing flow of I/O requests are listed as following:

a) The I/O handler of the hypervisor will fill an I/O request with
   PENDING state when a trapped I/O access happens in a User VM.
b) The hypervisor makes an upcall, which is a notification interrupt, to
   the Service VM.
c) The upcall handler schedules a tasklet to dispatch I/O requests.
d) The tasklet looks for the PENDING I/O requests, assigns them to
   different registered clients based on the address of the I/O accesses,
   updates their state to PROCESSING, and notifies the corresponding
   client to handle.
e) The notified client handles the assigned I/O requests.
f) The HSM updates I/O requests states to COMPLETE and notifies the
   hypervisor of the completion via hypercalls.

Signed-off-by: Shuo Liu 
Reviewed-by: Zhi Wang 
Reviewed-by: Reinette Chatre 
Cc: Zhi Wang 
Cc: Zhenyu Wang 
Cc: Yu Wang 
Cc: Reinette Chatre 
Cc: Greg Kroah-Hartman 
---
 drivers/virt/acrn/Makefile|   2 +-
 drivers/virt/acrn/acrn_drv.h  |  82 ++
 drivers/virt/acrn/hsm.c   |  34 ++-
 drivers/virt/acrn/hypercall.h |  28 ++
 drivers/virt/acrn/ioreq.c | 509 ++
 drivers/virt/acrn/vm.c|  27 +-
 include/uapi/linux/acrn.h | 134 +
 7 files changed, 806 insertions(+), 10 deletions(-)
 create mode 100644 drivers/virt/acrn/ioreq.c

diff --git a/drivers/virt/acrn/Makefile b/drivers/virt/acrn/Makefile
index 38bc44b6edcd..21721cbf6a80 100644
--- a/drivers/virt/acrn/Makefile
+++ b/drivers/virt/acrn/Makefile
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
 obj-$(CONFIG_ACRN_HSM) := acrn.o
-acrn-y := hsm.o vm.o mm.o
+acrn-y := hsm.o vm.o mm.o ioreq.o
diff --git a/drivers/virt/acrn/acrn_drv.h b/drivers/virt/acrn/acrn_drv.h
index e47a45280eea..7e89e4944d0f 100644
--- a/drivers/virt/acrn/acrn_drv.h
+++ b/drivers/virt/acrn/acrn_drv.h
@@ -12,10 +12,15 @@
 
 extern struct miscdevice acrn_dev;
 
+#define ACRN_NAME_LEN  16
 #define ACRN_MEM_MAPPING_MAX   256
 
 #define ACRN_MEM_REGION_ADD0
 #define ACRN_MEM_REGION_DEL2
+
+struct acrn_vm;
+struct acrn_ioreq_client;
+
 /**
  * struct vm_memory_region_op - Hypervisor memory operation
  * @type:  Operation type (ACRN_MEM_REGION_*)
@@ -75,9 +80,63 @@ struct vm_memory_mapping {
size_t  size;
 };
 
+/**
+ * struct acrn_ioreq_buffer - Data for setting the ioreq buffer of User VM
+ * @ioreq_buf: The GPA of the IO request shared buffer of a VM
+ *
+ * The parameter for the HC_SET_IOREQ_BUFFER hypercall used to set up
+ * the shared I/O request buffer between Service VM and ACRN hypervisor.
+ */
+struct acrn_ioreq_buffer {
+   u64 ioreq_buf;
+};
+
+struct acrn_ioreq_range {
+   struct list_headlist;
+   u32 type;
+   u64 start;
+   u64 end;
+};
+
+#define ACRN_IOREQ_CLIENT_DESTROYING   0U
+typedefint (*ioreq_handler_t)(struct acrn_ioreq_client *client,
+  struct acrn_io_request *req);
+/**
+ * struct acrn_ioreq_client - Structure of I/O client.
+ * @name:  Client name
+ * @vm:The VM that the client belongs to
+ * @list:  List node for this acrn_ioreq_client
+ * @is_default:If this client is the default one
+ * @flags: Flags