Re: [PATCH net-next v2 2/3] virtio-net: support IFF_TX_SKB_NO_LINEAR

[Michael S. Tsirkin]

On Wed, Jan 20, 2021 at 03:49:10PM +0800, Xuan Zhuo wrote:
> Virtio net supports the case where the skb linear space is empty, so add
> priv_flags.
> 
> Signed-off-by: Xuan Zhuo 

Acked-by: Michael S. Tsirkin 

> ---
>  drivers/net/virtio_net.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index ba8e637..f2ff6c3 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -2972,7 +2972,8 @@ static int virtnet_probe(struct virtio_device *vdev)
>   return -ENOMEM;
>  
>   /* Set up network device as normal. */
> - dev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE;
> + dev->priv_flags |= IFF_UNICAST_FLT | IFF_LIVE_ADDR_CHANGE |
> +IFF_TX_SKB_NO_LINEAR;
>   dev->netdev_ops = _netdev;
>   dev->features = NETIF_F_HIGHDMA;
>  
> -- 
> 1.8.3.1

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v1] vdpa/mlx5: Fix memory key MTT population

[Michael S. Tsirkin]

On Wed, Jan 20, 2021 at 07:36:19AM +0200, Eli Cohen wrote:
> On Fri, Jan 08, 2021 at 04:38:55PM +0800, Jason Wang wrote:
> 
> Hi Michael,
> this patch is a fix. Are you going to merge it?

yes - in the next pull request.

> > 
> > On 2021/1/7 下午3:18, Eli Cohen wrote:
> > > map_direct_mr() assumed that the number of scatter/gather entries
> > > returned by dma_map_sg_attrs() was equal to the number of segments in
> > > the sgl list. This led to wrong population of the mkey object. Fix this
> > > by properly referring to the returned value.
> > > 
> > > The hardware expects each MTT entry to contain the DMA address of a
> > > contiguous block of memory of size (1 << mr->log_size) bytes.
> > > dma_map_sg_attrs() can coalesce several sg entries into a single
> > > scatter/gather entry of contiguous DMA range so we need to scan the list
> > > and refer to the size of each s/g entry.
> > > 
> > > In addition, get rid of fill_sg() which effect is overwritten by
> > > populate_mtts().
> > > 
> > > Fixes: 94abbccdf291 ("vdpa/mlx5: Add shared memory registration code")
> > > Signed-off-by: Eli Cohen 
> > > ---
> > > V0->V1:
> > > 1. Fix typos
> > > 2. Improve changelog
> > 
> > 
> > Acked-by: Jason Wang 
> > 
> > 
> > > 
> > >   drivers/vdpa/mlx5/core/mlx5_vdpa.h |  1 +
> > >   drivers/vdpa/mlx5/core/mr.c| 28 
> > >   2 files changed, 13 insertions(+), 16 deletions(-)
> > > 
> > > diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h 
> > > b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> > > index 5c92a576edae..08f742fd2409 100644
> > > --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> > > +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h
> > > @@ -15,6 +15,7 @@ struct mlx5_vdpa_direct_mr {
> > >   struct sg_table sg_head;
> > >   int log_size;
> > >   int nsg;
> > > + int nent;
> > >   struct list_head list;
> > >   u64 offset;
> > >   };
> > > diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c
> > > index 4b6195666c58..d300f799efcd 100644
> > > --- a/drivers/vdpa/mlx5/core/mr.c
> > > +++ b/drivers/vdpa/mlx5/core/mr.c
> > > @@ -25,17 +25,6 @@ static int get_octo_len(u64 len, int page_shift)
> > >   return (npages + 1) / 2;
> > >   }
> > > -static void fill_sg(struct mlx5_vdpa_direct_mr *mr, void *in)
> > > -{
> > > - struct scatterlist *sg;
> > > - __be64 *pas;
> > > - int i;
> > > -
> > > - pas = MLX5_ADDR_OF(create_mkey_in, in, klm_pas_mtt);
> > > - for_each_sg(mr->sg_head.sgl, sg, mr->nsg, i)
> > > - (*pas) = cpu_to_be64(sg_dma_address(sg));
> > > -}
> > > -
> > >   static void mlx5_set_access_mode(void *mkc, int mode)
> > >   {
> > >   MLX5_SET(mkc, mkc, access_mode_1_0, mode & 0x3);
> > > @@ -45,10 +34,18 @@ static void mlx5_set_access_mode(void *mkc, int mode)
> > >   static void populate_mtts(struct mlx5_vdpa_direct_mr *mr, __be64 *mtt)
> > >   {
> > >   struct scatterlist *sg;
> > > + int nsg = mr->nsg;
> > > + u64 dma_addr;
> > > + u64 dma_len;
> > > + int j = 0;
> > >   int i;
> > > - for_each_sg(mr->sg_head.sgl, sg, mr->nsg, i)
> > > - mtt[i] = cpu_to_be64(sg_dma_address(sg));
> > > + for_each_sg(mr->sg_head.sgl, sg, mr->nent, i) {
> > > + for (dma_addr = sg_dma_address(sg), dma_len = sg_dma_len(sg);
> > > +  nsg && dma_len;
> > > +  nsg--, dma_addr += BIT(mr->log_size), dma_len -= 
> > > BIT(mr->log_size))
> > > + mtt[j++] = cpu_to_be64(dma_addr);
> > > + }
> > >   }
> > >   static int create_direct_mr(struct mlx5_vdpa_dev *mvdev, struct 
> > > mlx5_vdpa_direct_mr *mr)
> > > @@ -64,7 +61,6 @@ static int create_direct_mr(struct mlx5_vdpa_dev 
> > > *mvdev, struct mlx5_vdpa_direct
> > >   return -ENOMEM;
> > >   MLX5_SET(create_mkey_in, in, uid, mvdev->res.uid);
> > > - fill_sg(mr, in);
> > >   mkc = MLX5_ADDR_OF(create_mkey_in, in, memory_key_mkey_entry);
> > >   MLX5_SET(mkc, mkc, lw, !!(mr->perm & VHOST_MAP_WO));
> > >   MLX5_SET(mkc, mkc, lr, !!(mr->perm & VHOST_MAP_RO));
> > > @@ -276,8 +272,8 @@ static int map_direct_mr(struct mlx5_vdpa_dev *mvdev, 
> > > struct mlx5_vdpa_direct_mr
> > >   done:
> > >   mr->log_size = log_entity_size;
> > >   mr->nsg = nsg;
> > > - err = dma_map_sg_attrs(dma, mr->sg_head.sgl, mr->nsg, 
> > > DMA_BIDIRECTIONAL, 0);
> > > - if (!err)
> > > + mr->nent = dma_map_sg_attrs(dma, mr->sg_head.sgl, mr->nsg, 
> > > DMA_BIDIRECTIONAL, 0);
> > > + if (!mr->nent)
> > >   goto err_map;
> > >   err = create_direct_mr(mvdev, mr);
> > 

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices

[Michael S. Tsirkin]

On Mon, Jan 18, 2021 at 06:03:57PM +, Parav Pandit wrote:
> Hi Michael, Jason,
> 
> > From: Jason Wang 
> > Sent: Friday, January 15, 2021 11:09 AM
> > 
> > 
> > Thanks for the clarification. I think we'd better document the above in the
> > patch that introduces the mac setting from management API.
> 
> Can we proceed with this patchset?
> We like to progress next to iproute2/vdpa, mac and other drivers post this 
> series in this kernel version.

Let me put this in next so it can get some testing there for a week or
so.

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v3 06/11] vhost-vdpa: Add an opaque pointer for vhost IOTLB

[Jason Wang]

On 2021/1/19 下午12:59, Xie Yongji wrote:

Add an opaque pointer for vhost IOTLB to store the
corresponding vma->vm_file and offset on the DMA mapping.



Let's split the patch into two.

1) opaque pointer
2) vma stuffs




It will be used in VDUSE case later.

Suggested-by: Jason Wang 
Signed-off-by: Xie Yongji 
---
  drivers/vdpa/vdpa_sim/vdpa_sim.c | 11 ---
  drivers/vhost/iotlb.c|  5 ++-
  drivers/vhost/vdpa.c | 66 +++-
  drivers/vhost/vhost.c|  4 +--
  include/linux/vdpa.h |  3 +-
  include/linux/vhost_iotlb.h  |  8 -
  6 files changed, 79 insertions(+), 18 deletions(-)

diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 03c796873a6b..1ffcef67954f 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -279,7 +279,7 @@ static dma_addr_t vdpasim_map_page(struct device *dev, 
struct page *page,
 */
spin_lock(>iommu_lock);
ret = vhost_iotlb_add_range(iommu, pa, pa + size - 1,
-   pa, dir_to_perm(dir));
+   pa, dir_to_perm(dir), NULL);



Maybe its better to introduce

vhost_iotlb_add_range_ctx() which can accepts the opaque (context). And 
let vhost_iotlb_add_range() just call that.




spin_unlock(>iommu_lock);
if (ret)
return DMA_MAPPING_ERROR;
@@ -317,7 +317,7 @@ static void *vdpasim_alloc_coherent(struct device *dev, 
size_t size,
  
  		ret = vhost_iotlb_add_range(iommu, (u64)pa,

(u64)pa + size - 1,
-   pa, VHOST_MAP_RW);
+   pa, VHOST_MAP_RW, NULL);
if (ret) {
*dma_addr = DMA_MAPPING_ERROR;
kfree(addr);
@@ -625,7 +625,8 @@ static int vdpasim_set_map(struct vdpa_device *vdpa,
for (map = vhost_iotlb_itree_first(iotlb, start, last); map;
 map = vhost_iotlb_itree_next(map, start, last)) {
ret = vhost_iotlb_add_range(vdpasim->iommu, map->start,
-   map->last, map->addr, map->perm);
+   map->last, map->addr,
+   map->perm, NULL);
if (ret)
goto err;
}
@@ -639,14 +640,14 @@ static int vdpasim_set_map(struct vdpa_device *vdpa,
  }
  
  static int vdpasim_dma_map(struct vdpa_device *vdpa, u64 iova, u64 size,

-  u64 pa, u32 perm)
+  u64 pa, u32 perm, void *opaque)
  {
struct vdpasim *vdpasim = vdpa_to_sim(vdpa);
int ret;
  
  	spin_lock(>iommu_lock);

ret = vhost_iotlb_add_range(vdpasim->iommu, iova, iova + size - 1, pa,
-   perm);
+   perm, NULL);
spin_unlock(>iommu_lock);
  
  	return ret;

diff --git a/drivers/vhost/iotlb.c b/drivers/vhost/iotlb.c
index 0fd3f87e913c..3bd5bd06cdbc 100644
--- a/drivers/vhost/iotlb.c
+++ b/drivers/vhost/iotlb.c
@@ -42,13 +42,15 @@ EXPORT_SYMBOL_GPL(vhost_iotlb_map_free);
   * @last: last of IOVA range
   * @addr: the address that is mapped to @start
   * @perm: access permission of this range
+ * @opaque: the opaque pointer for the IOTLB mapping
   *
   * Returns an error last is smaller than start or memory allocation
   * fails
   */
  int vhost_iotlb_add_range(struct vhost_iotlb *iotlb,
  u64 start, u64 last,
- u64 addr, unsigned int perm)
+ u64 addr, unsigned int perm,
+ void *opaque)
  {
struct vhost_iotlb_map *map;
  
@@ -71,6 +73,7 @@ int vhost_iotlb_add_range(struct vhost_iotlb *iotlb,

map->last = last;
map->addr = addr;
map->perm = perm;
+   map->opaque = opaque;
  
  	iotlb->nmaps++;

vhost_iotlb_itree_insert(map, >root);
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 36b6950ba37f..e83e5be7cec8 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -488,6 +488,7 @@ static void vhost_vdpa_iotlb_unmap(struct vhost_vdpa *v, 
u64 start, u64 last)
struct vhost_dev *dev = >vdev;
struct vdpa_device *vdpa = v->vdpa;
struct vhost_iotlb *iotlb = dev->iotlb;
+   struct vhost_iotlb_file *iotlb_file;
struct vhost_iotlb_map *map;
struct page *page;
unsigned long pfn, pinned;
@@ -504,6 +505,10 @@ static void vhost_vdpa_iotlb_unmap(struct vhost_vdpa *v, 
u64 start, u64 last)
}
atomic64_sub(map->size >> PAGE_SHIFT,
>mm->pinned_vm);
+   } else if (map->opaque) {
+   iotlb_file = (struct vhost_iotlb_file *)map->opaque;
+   

Re: [RFC v3 05/11] vdpa: shared virtual addressing support

[Jason Wang]

On 2021/1/19 下午12:59, Xie Yongji wrote:

This patches introduces SVA (Shared Virtual Addressing)
support for vDPA device. During vDPA device allocation,
vDPA device driver needs to indicate whether SVA is
supported by the device. Then vhost-vdpa bus driver
will not pin user page and transfer userspace virtual
address instead of physical address during DMA mapping.

Suggested-by: Jason Wang 
Signed-off-by: Xie Yongji 
---
  drivers/vdpa/ifcvf/ifcvf_main.c   |  2 +-
  drivers/vdpa/mlx5/net/mlx5_vnet.c |  2 +-
  drivers/vdpa/vdpa.c   |  5 -
  drivers/vdpa/vdpa_sim/vdpa_sim.c  |  3 ++-
  drivers/vhost/vdpa.c  | 35 +++
  include/linux/vdpa.h  | 10 +++---
  6 files changed, 38 insertions(+), 19 deletions(-)

diff --git a/drivers/vdpa/ifcvf/ifcvf_main.c b/drivers/vdpa/ifcvf/ifcvf_main.c
index 23474af7da40..95c4601f82f5 100644
--- a/drivers/vdpa/ifcvf/ifcvf_main.c
+++ b/drivers/vdpa/ifcvf/ifcvf_main.c
@@ -439,7 +439,7 @@ static int ifcvf_probe(struct pci_dev *pdev, const struct 
pci_device_id *id)
  
  	adapter = vdpa_alloc_device(struct ifcvf_adapter, vdpa,

dev, _vdpa_ops,
-   IFCVF_MAX_QUEUE_PAIRS * 2, NULL);
+   IFCVF_MAX_QUEUE_PAIRS * 2, NULL, false);
if (adapter == NULL) {
IFCVF_ERR(pdev, "Failed to allocate vDPA structure");
return -ENOMEM;
diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c 
b/drivers/vdpa/mlx5/net/mlx5_vnet.c
index 77595c81488d..05988d6907f2 100644
--- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
+++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
@@ -1959,7 +1959,7 @@ static int mlx5v_probe(struct auxiliary_device *adev,
max_vqs = min_t(u32, max_vqs, MLX5_MAX_SUPPORTED_VQS);
  
  	ndev = vdpa_alloc_device(struct mlx5_vdpa_net, mvdev.vdev, mdev->device, _vdpa_ops,

-2 * mlx5_vdpa_max_qps(max_vqs), NULL);
+2 * mlx5_vdpa_max_qps(max_vqs), NULL, false);
if (IS_ERR(ndev))
return PTR_ERR(ndev);
  
diff --git a/drivers/vdpa/vdpa.c b/drivers/vdpa/vdpa.c

index 32bd48baffab..50cab930b2e5 100644
--- a/drivers/vdpa/vdpa.c
+++ b/drivers/vdpa/vdpa.c
@@ -72,6 +72,7 @@ static void vdpa_release_dev(struct device *d)
   * @nvqs: number of virtqueues supported by this device
   * @size: size of the parent structure that contains private data
   * @name: name of the vdpa device; optional.
+ * @sva: indicate whether SVA (Shared Virtual Addressing) is supported
   *
   * Driver should use vdpa_alloc_device() wrapper macro instead of
   * using this directly.
@@ -81,7 +82,8 @@ static void vdpa_release_dev(struct device *d)
   */
  struct vdpa_device *__vdpa_alloc_device(struct device *parent,
const struct vdpa_config_ops *config,
-   int nvqs, size_t size, const char *name)
+   int nvqs, size_t size, const char *name,
+   bool sva)
  {
struct vdpa_device *vdev;
int err = -EINVAL;
@@ -108,6 +110,7 @@ struct vdpa_device *__vdpa_alloc_device(struct device 
*parent,
vdev->config = config;
vdev->features_valid = false;
vdev->nvqs = nvqs;
+   vdev->sva = sva;
  
  	if (name)

err = dev_set_name(>dev, "%s", name);
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c
index 85776e4e6749..03c796873a6b 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c
@@ -367,7 +367,8 @@ static struct vdpasim *vdpasim_create(const char *name)
else
ops = _net_config_ops;
  
-	vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops, VDPASIM_VQ_NUM, name);

+   vdpasim = vdpa_alloc_device(struct vdpasim, vdpa, NULL, ops,
+   VDPASIM_VQ_NUM, name, false);
if (!vdpasim)
goto err_alloc;
  
diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c

index 4a241d380c40..36b6950ba37f 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -486,21 +486,25 @@ static long vhost_vdpa_unlocked_ioctl(struct file *filep,
  static void vhost_vdpa_iotlb_unmap(struct vhost_vdpa *v, u64 start, u64 last)
  {
struct vhost_dev *dev = >vdev;
+   struct vdpa_device *vdpa = v->vdpa;
struct vhost_iotlb *iotlb = dev->iotlb;
struct vhost_iotlb_map *map;
struct page *page;
unsigned long pfn, pinned;
  
  	while ((map = vhost_iotlb_itree_first(iotlb, start, last)) != NULL) {

-   pinned = map->size >> PAGE_SHIFT;
-   for (pfn = map->addr >> PAGE_SHIFT;
-pinned > 0; pfn++, pinned--) {
-   page = pfn_to_page(pfn);
-   if (map->perm & VHOST_ACCESS_WO)
-   

Re: [RFC v3 01/11] eventfd: track eventfd_signal() recursion depth separately in different cases

[Jason Wang]

On 2021/1/19 下午12:59, Xie Yongji wrote:

Now we have a global percpu counter to limit the recursion depth
of eventfd_signal(). This can avoid deadlock or stack overflow.
But in stack overflow case, it should be OK to increase the
recursion depth if needed. So we add a percpu counter in eventfd_ctx
to limit the recursion depth for deadlock case. Then it could be
fine to increase the global percpu counter later.



I wonder whether or not it's worth to introduce percpu for each eventfd.

How about simply check if eventfd_signal_count() is greater than 2?

Thanks




Signed-off-by: Xie Yongji 
---
  fs/aio.c|  3 ++-
  fs/eventfd.c| 20 +++-
  include/linux/eventfd.h |  5 +
  3 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/fs/aio.c b/fs/aio.c
index 1f32da13d39e..5d82903161f5 100644
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -1698,7 +1698,8 @@ static int aio_poll_wake(struct wait_queue_entry *wait, 
unsigned mode, int sync,
list_del(>ki_list);
iocb->ki_res.res = mangle_poll(mask);
req->done = true;
-   if (iocb->ki_eventfd && eventfd_signal_count()) {
+   if (iocb->ki_eventfd &&
+   eventfd_signal_count(iocb->ki_eventfd)) {
iocb = NULL;
INIT_WORK(>work, aio_poll_put_work);
schedule_work(>work);
diff --git a/fs/eventfd.c b/fs/eventfd.c
index e265b6dd4f34..2df24f9bada3 100644
--- a/fs/eventfd.c
+++ b/fs/eventfd.c
@@ -25,6 +25,8 @@
  #include 
  #include 
  
+#define EVENTFD_WAKE_DEPTH 0

+
  DEFINE_PER_CPU(int, eventfd_wake_count);
  
  static DEFINE_IDA(eventfd_ida);

@@ -42,9 +44,17 @@ struct eventfd_ctx {
 */
__u64 count;
unsigned int flags;
+   int __percpu *wake_count;
int id;
  };
  
+bool eventfd_signal_count(struct eventfd_ctx *ctx)

+{
+   return (this_cpu_read(*ctx->wake_count) ||
+   this_cpu_read(eventfd_wake_count) > EVENTFD_WAKE_DEPTH);
+}
+EXPORT_SYMBOL_GPL(eventfd_signal_count);
+
  /**
   * eventfd_signal - Adds @n to the eventfd counter.
   * @ctx: [in] Pointer to the eventfd context.
@@ -71,17 +81,19 @@ __u64 eventfd_signal(struct eventfd_ctx *ctx, __u64 n)
 * it returns true, the eventfd_signal() call should be deferred to a
 * safe context.
 */
-   if (WARN_ON_ONCE(this_cpu_read(eventfd_wake_count)))
+   if (WARN_ON_ONCE(eventfd_signal_count(ctx)))
return 0;
  
  	spin_lock_irqsave(>wqh.lock, flags);

this_cpu_inc(eventfd_wake_count);
+   this_cpu_inc(*ctx->wake_count);
if (ULLONG_MAX - ctx->count < n)
n = ULLONG_MAX - ctx->count;
ctx->count += n;
if (waitqueue_active(>wqh))
wake_up_locked_poll(>wqh, EPOLLIN);
this_cpu_dec(eventfd_wake_count);
+   this_cpu_dec(*ctx->wake_count);
spin_unlock_irqrestore(>wqh.lock, flags);
  
  	return n;

@@ -92,6 +104,7 @@ static void eventfd_free_ctx(struct eventfd_ctx *ctx)
  {
if (ctx->id >= 0)
ida_simple_remove(_ida, ctx->id);
+   free_percpu(ctx->wake_count);
kfree(ctx);
  }
  
@@ -423,6 +436,11 @@ static int do_eventfd(unsigned int count, int flags)
  
  	kref_init(>kref);

init_waitqueue_head(>wqh);
+   ctx->wake_count = alloc_percpu(int);
+   if (!ctx->wake_count) {
+   kfree(ctx);
+   return -ENOMEM;
+   }
ctx->count = count;
ctx->flags = flags;
ctx->id = ida_simple_get(_ida, 0, 0, GFP_KERNEL);
diff --git a/include/linux/eventfd.h b/include/linux/eventfd.h
index fa0a524baed0..1a11ebbd74a9 100644
--- a/include/linux/eventfd.h
+++ b/include/linux/eventfd.h
@@ -45,10 +45,7 @@ void eventfd_ctx_do_read(struct eventfd_ctx *ctx, __u64 
*cnt);
  
  DECLARE_PER_CPU(int, eventfd_wake_count);
  
-static inline bool eventfd_signal_count(void)

-{
-   return this_cpu_read(eventfd_wake_count);
-}
+bool eventfd_signal_count(struct eventfd_ctx *ctx);
  
  #else /* CONFIG_EVENTFD */
  


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v3 03/11] vdpa: Remove the restriction that only supports virtio-net devices

[Jason Wang]

On 2021/1/19 下午12:59, Xie Yongji wrote:

With VDUSE, we should be able to support all kinds of virtio devices.

Signed-off-by: Xie Yongji 
---
  drivers/vhost/vdpa.c | 29 +++--
  1 file changed, 3 insertions(+), 26 deletions(-)

diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 29ed4173f04e..448be7875b6d 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -22,6 +22,7 @@
  #include 
  #include 
  #include 
+#include 
  
  #include "vhost.h"
  
@@ -185,26 +186,6 @@ static long vhost_vdpa_set_status(struct vhost_vdpa *v, u8 __user *statusp)

return 0;
  }
  
-static int vhost_vdpa_config_validate(struct vhost_vdpa *v,

- struct vhost_vdpa_config *c)
-{
-   long size = 0;
-
-   switch (v->virtio_id) {
-   case VIRTIO_ID_NET:
-   size = sizeof(struct virtio_net_config);
-   break;
-   }
-
-   if (c->len == 0)
-   return -EINVAL;
-
-   if (c->len > size - c->off)
-   return -E2BIG;
-
-   return 0;
-}



I think we should use a separate patch for this.

Thanks



-
  static long vhost_vdpa_get_config(struct vhost_vdpa *v,
  struct vhost_vdpa_config __user *c)
  {
@@ -215,7 +196,7 @@ static long vhost_vdpa_get_config(struct vhost_vdpa *v,
  
  	if (copy_from_user(, c, size))

return -EFAULT;
-   if (vhost_vdpa_config_validate(v, ))
+   if (config.len == 0)
return -EINVAL;
buf = kvzalloc(config.len, GFP_KERNEL);
if (!buf)
@@ -243,7 +224,7 @@ static long vhost_vdpa_set_config(struct vhost_vdpa *v,
  
  	if (copy_from_user(, c, size))

return -EFAULT;
-   if (vhost_vdpa_config_validate(v, ))
+   if (config.len == 0)
return -EINVAL;
buf = kvzalloc(config.len, GFP_KERNEL);
if (!buf)
@@ -1025,10 +1006,6 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)
int minor;
int r;
  
-	/* Currently, we only accept the network devices. */

-   if (ops->get_device_id(vdpa) != VIRTIO_ID_NET)
-   return -ENOTSUPP;
-
v = kzalloc(sizeof(*v), GFP_KERNEL | __GFP_RETRY_MAYFAIL);
if (!v)
return -ENOMEM;


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

RE: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices

[Parav Pandit]

> From: Virtualization  On
> Behalf Of Parav Pandit
> 
> > From: Jason Wang 
> > Sent: Tuesday, January 19, 2021 4:39 PM
> > To: Parav Pandit ; Michael S. Tsirkin
> > 
> > Cc: virtualization@lists.linux-foundation.org; Eli Cohen
> > ; net...@vger.kernel.org; Sean Mooney
> > 
> > Subject: Re: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for
> > user supported devices
> >
> >
> > On 2021/1/15 下午2:27, Parav Pandit wrote:
> > >>> With that mac, mtu as optional input fields provide the necessary
> > >>> flexibility
> > >> for different stacks to take appropriate shape as they desire.
> > >>
> > >>
> > >> Thanks for the clarification. I think we'd better document the
> > >> above in the patch that introduces the mac setting from management
> API.
> > > Yes. Will do.
> > > Thanks.
> >
> >
> > Adding Sean.
> >
> > Regarding to mac address setting. Do we plan to allow to modify mac
> > address after the creation? It looks like Openstack wants this.
> >
> Mac address is exposed in the features so yes, it should be possible to
> modify it as part of features modify command. (in future).
> User needs to make sure that device is not attached to vhost or higher layer
> stack when device configuration layout is modified.
> 
Just curious, why Openstack cannot set the mac address at device creation time?
Is vdpa device created by non Openstack software?
Does it always want to set the mac address of vdpa device?

> > Sean may share more information on this.
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v3 04/11] vhost-vdpa: protect concurrent access to vhost device iotlb

[Jason Wang]

On 2021/1/19 下午12:59, Xie Yongji wrote:

Introduce a mutex to protect vhost device iotlb from
concurrent access.

Fixes: 4c8cf318("vhost: introduce vDPA-based backend")
Signed-off-by: Xie Yongji 
---
  drivers/vhost/vdpa.c | 4 
  1 file changed, 4 insertions(+)

diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
index 448be7875b6d..4a241d380c40 100644
--- a/drivers/vhost/vdpa.c
+++ b/drivers/vhost/vdpa.c
@@ -49,6 +49,7 @@ struct vhost_vdpa {
struct eventfd_ctx *config_ctx;
int in_batch;
struct vdpa_iova_range range;
+   struct mutex mutex;



Let's use the device mutex like what vhost_process_iotlb_msg() did.

Thanks



  };
  
  static DEFINE_IDA(vhost_vdpa_ida);

@@ -728,6 +729,7 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev 
*dev,
if (r)
return r;
  
+	mutex_lock(>mutex);

switch (msg->type) {
case VHOST_IOTLB_UPDATE:
r = vhost_vdpa_process_iotlb_update(v, msg);
@@ -747,6 +749,7 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev 
*dev,
r = -EINVAL;
break;
}
+   mutex_unlock(>mutex);
  
  	return r;

  }
@@ -1017,6 +1020,7 @@ static int vhost_vdpa_probe(struct vdpa_device *vdpa)
return minor;
}
  
+	mutex_init(>mutex);

atomic_set(>opened, 0);
v->minor = minor;
v->vdpa = vdpa;


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH net-next v2 5/7] virtio-net, xsk: realize the function of xsk packet sending

[Jason Wang]

On 2021/1/18 下午8:03, Xuan Zhuo wrote:

On Mon, 18 Jan 2021 17:10:24 +0800, Jason Wang  wrote:

On 2021/1/16 上午10:59, Xuan Zhuo wrote:

virtnet_xsk_run will be called in the tx interrupt handling function
virtnet_poll_tx.

The sending process gets desc from the xsk tx queue, and assembles it to
send the data.

Compared with other drivers, a special place is that the page of the
data in xsk is used here instead of the dma address. Because the virtio
interface does not use the dma address.

Signed-off-by: Xuan Zhuo 
---
   drivers/net/virtio_net.c | 200 
++-
   1 file changed, 197 insertions(+), 3 deletions(-)

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a62d456..42aa9ad 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -119,6 +119,8 @@ struct virtnet_xsk_hdr {
u32 len;
   };

+#define VIRTNET_STATE_XSK_WAKEUP 1
+
   #define VIRTNET_SQ_STAT(m)   offsetof(struct virtnet_sq_stats, m)
   #define VIRTNET_RQ_STAT(m)   offsetof(struct virtnet_rq_stats, m)

@@ -163,9 +165,12 @@ struct send_queue {
struct xsk_buff_pool   __rcu *pool;
struct virtnet_xsk_hdr __rcu *hdr;

+   unsigned long  state;
u64hdr_con;
u64hdr_pro;
u64hdr_n;
+   struct xdp_desclast_desc;
+   bool   wait_slot;
} xsk;
   };

@@ -284,6 +289,8 @@ static void __free_old_xmit_ptr(struct send_queue *sq, bool 
in_napi,
bool xsk_wakeup,
unsigned int *_packets, unsigned int *_bytes);
   static void free_old_xmit_skbs(struct send_queue *sq, bool in_napi);
+static int virtnet_xsk_run(struct send_queue *sq,
+  struct xsk_buff_pool *pool, int budget);

   static bool is_xdp_frame(void *ptr)
   {
@@ -1590,6 +1597,8 @@ static int virtnet_poll_tx(struct napi_struct *napi, int 
budget)
struct virtnet_info *vi = sq->vq->vdev->priv;
unsigned int index = vq2txq(sq->vq);
struct netdev_queue *txq;
+   struct xsk_buff_pool *pool;
+   int work = 0;

if (unlikely(is_xdp_raw_buffer_queue(vi, index))) {
/* We don't need to enable cb for XDP */
@@ -1599,15 +1608,26 @@ static int virtnet_poll_tx(struct napi_struct *napi, 
int budget)

txq = netdev_get_tx_queue(vi->dev, index);
__netif_tx_lock(txq, raw_smp_processor_id());
-   free_old_xmit_skbs(sq, true);
+
+   rcu_read_lock();
+   pool = rcu_dereference(sq->xsk.pool);
+   if (pool) {
+   work = virtnet_xsk_run(sq, pool, budget);
+   rcu_read_unlock();
+   } else {
+   rcu_read_unlock();
+   free_old_xmit_skbs(sq, true);
+   }
+
__netif_tx_unlock(txq);

-   virtqueue_napi_complete(napi, sq->vq, 0);
+   if (work < budget)
+   virtqueue_napi_complete(napi, sq->vq, 0);

if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS)
netif_tx_wake_queue(txq);

-   return 0;
+   return work;
   }

   static int xmit_skb(struct send_queue *sq, struct sk_buff *skb)
@@ -2647,6 +2667,180 @@ static int virtnet_xdp(struct net_device *dev, struct 
netdev_bpf *xdp)
}
   }

+static int virtnet_xsk_xmit(struct send_queue *sq, struct xsk_buff_pool *pool,
+   struct xdp_desc *desc)
+{
+   struct virtnet_info *vi = sq->vq->vdev->priv;
+   void *data, *ptr;
+   struct page *page;
+   struct virtnet_xsk_hdr *xskhdr;
+   u32 idx, offset, n, i, copy, copied;
+   u64 addr;
+   int err, m;
+
+   addr = desc->addr;
+
+   data = xsk_buff_raw_get_data(pool, addr);
+   offset = offset_in_page(data);
+
+   /* one for hdr, one for the first page */
+   n = 2;
+   m = desc->len - (PAGE_SIZE - offset);
+   if (m > 0) {
+   n += m >> PAGE_SHIFT;
+   if (m & PAGE_MASK)
+   ++n;
+
+   n = min_t(u32, n, ARRAY_SIZE(sq->sg));
+   }
+
+   idx = sq->xsk.hdr_con % sq->xsk.hdr_n;


I don't understand the reason of the hdr array. It looks to me all of
them are zero and read only from device.

Any reason for not reusing a single hdr for all xdp descriptors? Or
maybe it's time to introduce VIRTIO_NET_F_NO_HDR.

Yes, You are right.
Before supporting functions like csum, here it is indeed possible to achieve it
with only one hdr.



So let's drop the array logic for now since it will give unnecessary 
stress on the cache.






+   xskhdr = >xsk.hdr[idx];
+
+   /* xskhdr->hdr has been memset to zero, so not need to clear again */
+
+   sg_init_table(sq->sg, n);
+   sg_set_buf(sq->sg, >hdr, vi->hdr_len);
+
+   copied = 0;
+   for (i = 1; i < n; ++i) {
+   copy = min_t(int, desc->len - copied, PAGE_SIZE - 

Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Alex Williamson]

On Wed, 20 Jan 2021 03:05:49 +
"Tian, Kevin"  wrote:

> > From: Alex Williamson
> > Sent: Wednesday, January 20, 2021 8:51 AM
> > 
> > On Wed, 20 Jan 2021 00:14:49 +
> > "Kasireddy, Vivek"  wrote:
> >   
> > > Hi Alex,
> > >  
> > > > -Original Message-
> > > > From: Alex Williamson 
> > > > Sent: Tuesday, January 19, 2021 7:40 AM
> > > > To: Kasireddy, Vivek 
> > > > Cc: virtualization@lists.linux-foundation.org; Kim, Dongwon  
> >   
> > > > Subject: Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf
> > > >
> > > > On Tue, 19 Jan 2021 00:28:12 -0800
> > > > Vivek Kasireddy  wrote:
> > > >  
> > > > > Getting a copy of the KVM instance is necessary for mapping Guest
> > > > > pages in the Host.
> > > > >
> > > > > TODO: Instead of invoking the symbol directly, there needs to be a
> > > > > better way of getting a copy of the KVM instance probably by using
> > > > > other notifiers. However, currently, KVM shares its instance only
> > > > > with VFIO and therefore we are compelled to bind the passthrough'd
> > > > > device to vfio-pci.  
> > > >
> > > > Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
> > > > call out to vhost to share a kvm pointer.  I'd prefer to get rid of
> > > > vfio having any knowledge or visibility of the kvm pointer.  Thanks,  
> > >
> > > [Kasireddy, Vivek] I agree that this is definitely not ideal as I 
> > > recognize it
> > > in the TODO. However, it looks like VFIO also gets a copy of the KVM
> > > pointer in a similar manner:
> > >
> > > virt/kvm/vfio.c
> > >
> > > static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm  
> > *kvm)  
> > > {
> > > void (*fn)(struct vfio_group *, struct kvm *);
> > >
> > > fn = symbol_get(vfio_group_set_kvm);
> > > if (!fn)
> > > return;
> > >
> > > fn(group, kvm);
> > >
> > > symbol_put(vfio_group_set_kvm);
> > > }  
> > 
> > You're equating the mechanism with the architecture.  We use symbols
> > here to avoid module dependencies between kvm and vfio, but this is
> > just propagating data that userspace is specifically registering
> > between kvm and vfio.  vhost doesn't get to piggyback on that channel.
> >   
> > > With this patch, I am not suggesting that this is a precedent that should 
> > > be  
> > followed  
> > > but it appears there doesn't seem to be an alternative way of getting a 
> > > copy  
> > of the KVM  
> > > pointer that is clean and elegant -- unless I have not looked hard 
> > > enough. I  
> > guess we  
> > > could create a notifier chain with callbacks for VFIO and Vhost that KVM  
> > would call  
> > > but this would mean modifying KVM.
> > >
> > > Also, if I understand correctly, if VFIO does not want to share the KVM  
> > pointer with  
> > > VFIO groups, then I think it would break stuff like mdev which counts on 
> > > it.  
> > 
> > Only kvmgt requires the kvm pointer and the use case there is pretty
> > questionable, I wonder if it actually still exists now that we have the
> > DMA r/w interface through vfio.  Thanks,
> >   
> 
> IIRC, kvmgt still needs the kvm pointer to use kvm page tracking interface 
> for write-protecting guest pgtable.

Thanks, Kevin.  Either way, a vhost device has no stake in the game wrt
the kvm pointer lifecycle here and no business adding a callout.  I'm
reluctant to add any further use cases even for mdevs as ideally mdevs
should have no dependency on kvm.  Thanks,

Alex

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH net-next v2 2/7] virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx

[Jason Wang]

On 2021/1/18 下午7:54, Xuan Zhuo wrote:

On Mon, 18 Jan 2021 14:45:16 +0800, Jason Wang  wrote:

On 2021/1/16 上午10:59, Xuan Zhuo wrote:

If support xsk, a new ptr will be recovered during the
process of freeing the old ptr. In order to distinguish between ctx sent
by XDP_TX and ctx sent by xsk, a struct is added here to distinguish
between these two situations. virtnet_xdp_type.type It is used to
distinguish different ctx, and virtnet_xdp_type.offset is used to record
the offset between "true ctx" and virtnet_xdp_type.

The newly added virtnet_xsk_hdr will be used for xsk.

Signed-off-by: Xuan Zhuo 


Any reason that you can't simply encode the type in the pointer itself
as we used to do?

#define VIRTIO_XSK_FLAG    BIT(1)

?

Since xdp socket does not use xdp_frame, we will encounter three data types when
recycling: skb, xdp_frame, xdp socket.



Just to make sure we are in the same page. Currently, the pointer type 
is encoded with 1 bit in the pointer. Can we simply use 2 bit to 
distinguish skb, xdp, xsk?


Thanks






___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

RE: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices

[Parav Pandit]

> From: Jason Wang 
> Sent: Tuesday, January 19, 2021 4:39 PM
> To: Parav Pandit ; Michael S. Tsirkin 
> Cc: virtualization@lists.linux-foundation.org; Eli Cohen ;
> net...@vger.kernel.org; Sean Mooney 
> Subject: Re: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user
> supported devices
> 
> 
> On 2021/1/15 下午2:27, Parav Pandit wrote:
> >>> With that mac, mtu as optional input fields provide the necessary
> >>> flexibility
> >> for different stacks to take appropriate shape as they desire.
> >>
> >>
> >> Thanks for the clarification. I think we'd better document the above
> >> in the patch that introduces the mac setting from management API.
> > Yes. Will do.
> > Thanks.
> 
> 
> Adding Sean.
> 
> Regarding to mac address setting. Do we plan to allow to modify mac
> address after the creation? It looks like Openstack wants this.
>
Mac address is exposed in the features so yes, it should be possible to modify 
it as part of features modify command. (in future).
User needs to make sure that device is not attached to vhost or higher layer 
stack when device configuration layout is modified.
 
> Sean may share more information on this.
> 
> Thanks

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

RE: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Tian, Kevin]

> From: Alex Williamson
> Sent: Wednesday, January 20, 2021 8:51 AM
> 
> On Wed, 20 Jan 2021 00:14:49 +
> "Kasireddy, Vivek"  wrote:
> 
> > Hi Alex,
> >
> > > -Original Message-
> > > From: Alex Williamson 
> > > Sent: Tuesday, January 19, 2021 7:40 AM
> > > To: Kasireddy, Vivek 
> > > Cc: virtualization@lists.linux-foundation.org; Kim, Dongwon
> 
> > > Subject: Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf
> > >
> > > On Tue, 19 Jan 2021 00:28:12 -0800
> > > Vivek Kasireddy  wrote:
> > >
> > > > Getting a copy of the KVM instance is necessary for mapping Guest
> > > > pages in the Host.
> > > >
> > > > TODO: Instead of invoking the symbol directly, there needs to be a
> > > > better way of getting a copy of the KVM instance probably by using
> > > > other notifiers. However, currently, KVM shares its instance only
> > > > with VFIO and therefore we are compelled to bind the passthrough'd
> > > > device to vfio-pci.
> > >
> > > Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
> > > call out to vhost to share a kvm pointer.  I'd prefer to get rid of
> > > vfio having any knowledge or visibility of the kvm pointer.  Thanks,
> >
> > [Kasireddy, Vivek] I agree that this is definitely not ideal as I recognize 
> > it
> > in the TODO. However, it looks like VFIO also gets a copy of the KVM
> > pointer in a similar manner:
> >
> > virt/kvm/vfio.c
> >
> > static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm
> *kvm)
> > {
> > void (*fn)(struct vfio_group *, struct kvm *);
> >
> > fn = symbol_get(vfio_group_set_kvm);
> > if (!fn)
> > return;
> >
> > fn(group, kvm);
> >
> > symbol_put(vfio_group_set_kvm);
> > }
> 
> You're equating the mechanism with the architecture.  We use symbols
> here to avoid module dependencies between kvm and vfio, but this is
> just propagating data that userspace is specifically registering
> between kvm and vfio.  vhost doesn't get to piggyback on that channel.
> 
> > With this patch, I am not suggesting that this is a precedent that should be
> followed
> > but it appears there doesn't seem to be an alternative way of getting a copy
> of the KVM
> > pointer that is clean and elegant -- unless I have not looked hard enough. I
> guess we
> > could create a notifier chain with callbacks for VFIO and Vhost that KVM
> would call
> > but this would mean modifying KVM.
> >
> > Also, if I understand correctly, if VFIO does not want to share the KVM
> pointer with
> > VFIO groups, then I think it would break stuff like mdev which counts on it.
> 
> Only kvmgt requires the kvm pointer and the use case there is pretty
> questionable, I wonder if it actually still exists now that we have the
> DMA r/w interface through vfio.  Thanks,
> 

IIRC, kvmgt still needs the kvm pointer to use kvm page tracking interface 
for write-protecting guest pgtable.

Thanks
Kevin
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH bpf-next v2 1/3] net: add priv_flags for allow tx skb without linear

[Jason Wang]

On 2021/1/19 下午5:45, Xuan Zhuo wrote:

In some cases, we hope to construct skb directly based on the existing
memory without copying data. In this case, the page will be placed
directly in the skb, and the linear space of skb is empty. But
unfortunately, many the network card does not support this operation.
For example Mellanox Technologies MT27710 Family [ConnectX-4 Lx] will
get the following error message:

 mlx5_core :3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, 
opcode 0xd, syndrome 0x1, vendor syndrome 0x68
 : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 0030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2
 WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64
 : 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00
 0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 0020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00
 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 mlx5_core :3b:00.1 eth1: ERR CQE on SQ: 0x1dbb

So a priv_flag is added here to indicate whether the network card
supports this feature.



I don't see Mellanox engineers are copied. I wonder if we need their 
confirmation on whether it's a bug or hardware limitation.


Thanks

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH RFC] virtio-blk: support per-device queue depth

[Jason Wang]

On 2021/1/20 上午9:51, JeffleXu wrote:


On 1/19/21 12:06 PM, Jason Wang wrote:

On 2021/1/19 上午9:33, JeffleXu wrote:

On 1/18/21 1:25 PM, Jason Wang wrote:

On 2021/1/18 上午11:58, Joseph Qi wrote:

module parameter 'virtblk_queue_depth' was firstly introduced for
testing/benchmarking purposes described in commit fc4324b4597c
("virtio-blk: base queue-depth on virtqueue ringsize or module param").
Since we have different virtio-blk devices which have different
capabilities, it requires that we support per-device queue depth
instead
of per-module. So defaultly use vq free elements if module parameter
'virtblk_queue_depth' is not set.

I wonder if it's better to use sysfs instead (or whether it has already
had something like this in the blocker layer).


"/sys/block//queue/nr_requests" indeed works, but isn't better to
set queue_depth according to the hardware capability at the very first?
AFAIK, nvme just set per-device queue_depth at initializing phase.


I agree, the problem is that the current code may modify module parameter.

The module parameter 'virtblk_queue_depth' is actually remained untainted.

Actually it is the original code before this patch that changes the
module parameter.



Yes, that's what I meant.



When the module parameter is not set by boot cmdline
(i.e., default to 0), it will be initialized to the queue_depth of the
vring of the first probed virtio-blk device, and will be revealed to
user space through '/sys/module/virtio_blk/parameters/queue_depth'. I'm
not sure if this behavior is reasonable or not.



Right, it means the virtio-blk devices that is probed after the first 
one can only use the queue_depth that is set according to the capability 
of the first virtio-blk device.





The only side effect of this patch is that, now
'/sys/module/virtio_blk/parameters/queue_depth' will be kept as '0' when
the module parameter is not set manually.



I think it's not an issue, the nr_request should be the correct way to  
get per device queue depth.


Thanks





Thanks,
Jeffle



Signed-off-by: Joseph Qi 
---
    drivers/block/virtio_blk.c | 12 +++-
    1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index 145606d..f83a417 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -705,6 +705,7 @@ static int virtblk_probe(struct virtio_device
*vdev)
    u32 v, blk_size, max_size, sg_elems, opt_io_size;
    u16 min_io_size;
    u8 physical_block_exp, alignment_offset;
+    unsigned int queue_depth;
      if (!vdev->config->get) {
    dev_err(>dev, "%s failure: config access disabled\n",
@@ -755,17 +756,18 @@ static int virtblk_probe(struct virtio_device
*vdev)
    goto out_free_vq;
    }
    -    /* Default queue sizing is to fill the ring. */
-    if (!virtblk_queue_depth) {
-    virtblk_queue_depth = vblk->vqs[0].vq->num_free;
+    if (likely(!virtblk_queue_depth)) {
+    queue_depth = vblk->vqs[0].vq->num_free;
    /* ... but without indirect descs, we use 2 descs per req */
    if (!virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC))
-    virtblk_queue_depth /= 2;
+    queue_depth /= 2;
+    } else {
+    queue_depth = virtblk_queue_depth;
    }
      memset(>tag_set, 0, sizeof(vblk->tag_set));
    vblk->tag_set.ops = _mq_ops;
-    vblk->tag_set.queue_depth = virtblk_queue_depth;
+    vblk->tag_set.queue_depth = queue_depth;
    vblk->tag_set.numa_node = NUMA_NO_NODE;
    vblk->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
    vblk->tag_set.cmd_size =


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH RFC] virtio-blk: support per-device queue depth

[JeffleXu]

On 1/19/21 12:06 PM, Jason Wang wrote:
> 
> On 2021/1/19 上午9:33, JeffleXu wrote:
>>
>> On 1/18/21 1:25 PM, Jason Wang wrote:
>>> On 2021/1/18 上午11:58, Joseph Qi wrote:
 module parameter 'virtblk_queue_depth' was firstly introduced for
 testing/benchmarking purposes described in commit fc4324b4597c
 ("virtio-blk: base queue-depth on virtqueue ringsize or module param").
 Since we have different virtio-blk devices which have different
 capabilities, it requires that we support per-device queue depth
 instead
 of per-module. So defaultly use vq free elements if module parameter
 'virtblk_queue_depth' is not set.
>>>
>>> I wonder if it's better to use sysfs instead (or whether it has already
>>> had something like this in the blocker layer).
>>>
>> "/sys/block//queue/nr_requests" indeed works, but isn't better to
>> set queue_depth according to the hardware capability at the very first?
>> AFAIK, nvme just set per-device queue_depth at initializing phase.
> 
> 
> I agree, the problem is that the current code may modify module parameter.

The module parameter 'virtblk_queue_depth' is actually remained untainted.

Actually it is the original code before this patch that changes the
module parameter. When the module parameter is not set by boot cmdline
(i.e., default to 0), it will be initialized to the queue_depth of the
vring of the first probed virtio-blk device, and will be revealed to
user space through '/sys/module/virtio_blk/parameters/queue_depth'. I'm
not sure if this behavior is reasonable or not.

The only side effect of this patch is that, now
'/sys/module/virtio_blk/parameters/queue_depth' will be kept as '0' when
the module parameter is not set manually.


Thanks,
Jeffle


>>
>>>
 Signed-off-by: Joseph Qi 
 ---
    drivers/block/virtio_blk.c | 12 +++-
    1 file changed, 7 insertions(+), 5 deletions(-)

 diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
 index 145606d..f83a417 100644
 --- a/drivers/block/virtio_blk.c
 +++ b/drivers/block/virtio_blk.c
 @@ -705,6 +705,7 @@ static int virtblk_probe(struct virtio_device
 *vdev)
    u32 v, blk_size, max_size, sg_elems, opt_io_size;
    u16 min_io_size;
    u8 physical_block_exp, alignment_offset;
 +    unsigned int queue_depth;
      if (!vdev->config->get) {
    dev_err(>dev, "%s failure: config access disabled\n",
 @@ -755,17 +756,18 @@ static int virtblk_probe(struct virtio_device
 *vdev)
    goto out_free_vq;
    }
    -    /* Default queue sizing is to fill the ring. */
 -    if (!virtblk_queue_depth) {
 -    virtblk_queue_depth = vblk->vqs[0].vq->num_free;
 +    if (likely(!virtblk_queue_depth)) {
 +    queue_depth = vblk->vqs[0].vq->num_free;
    /* ... but without indirect descs, we use 2 descs per req */
    if (!virtio_has_feature(vdev, VIRTIO_RING_F_INDIRECT_DESC))
 -    virtblk_queue_depth /= 2;
 +    queue_depth /= 2;
 +    } else {
 +    queue_depth = virtblk_queue_depth;
    }
      memset(>tag_set, 0, sizeof(vblk->tag_set));
    vblk->tag_set.ops = _mq_ops;
 -    vblk->tag_set.queue_depth = virtblk_queue_depth;
 +    vblk->tag_set.queue_depth = queue_depth;
    vblk->tag_set.numa_node = NUMA_NO_NODE;
    vblk->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
    vblk->tag_set.cmd_size =

-- 
Thanks,
Jeffle
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Alex Williamson]

On Wed, 20 Jan 2021 00:14:49 +
"Kasireddy, Vivek"  wrote:

> Hi Alex,
> 
> > -Original Message-
> > From: Alex Williamson 
> > Sent: Tuesday, January 19, 2021 7:40 AM
> > To: Kasireddy, Vivek 
> > Cc: virtualization@lists.linux-foundation.org; Kim, Dongwon 
> > 
> > Subject: Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf
> > 
> > On Tue, 19 Jan 2021 00:28:12 -0800
> > Vivek Kasireddy  wrote:
> >   
> > > Getting a copy of the KVM instance is necessary for mapping Guest
> > > pages in the Host.
> > >
> > > TODO: Instead of invoking the symbol directly, there needs to be a
> > > better way of getting a copy of the KVM instance probably by using
> > > other notifiers. However, currently, KVM shares its instance only
> > > with VFIO and therefore we are compelled to bind the passthrough'd
> > > device to vfio-pci.  
> > 
> > Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
> > call out to vhost to share a kvm pointer.  I'd prefer to get rid of
> > vfio having any knowledge or visibility of the kvm pointer.  Thanks,  
> 
> [Kasireddy, Vivek] I agree that this is definitely not ideal as I recognize it
> in the TODO. However, it looks like VFIO also gets a copy of the KVM 
> pointer in a similar manner:
> 
> virt/kvm/vfio.c
> 
> static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
> {
> void (*fn)(struct vfio_group *, struct kvm *);
> 
> fn = symbol_get(vfio_group_set_kvm);
> if (!fn)
> return;
> 
> fn(group, kvm);
> 
> symbol_put(vfio_group_set_kvm);
> }

You're equating the mechanism with the architecture.  We use symbols
here to avoid module dependencies between kvm and vfio, but this is
just propagating data that userspace is specifically registering
between kvm and vfio.  vhost doesn't get to piggyback on that channel.

> With this patch, I am not suggesting that this is a precedent that should be 
> followed 
> but it appears there doesn't seem to be an alternative way of getting a copy 
> of the KVM 
> pointer that is clean and elegant -- unless I have not looked hard enough. I 
> guess we
> could create a notifier chain with callbacks for VFIO and Vhost that KVM 
> would call 
> but this would mean modifying KVM.
> 
> Also, if I understand correctly, if VFIO does not want to share the KVM 
> pointer with
> VFIO groups, then I think it would break stuff like mdev which counts on it. 

Only kvmgt requires the kvm pointer and the use case there is pretty
questionable, I wonder if it actually still exists now that we have the
DMA r/w interface through vfio.  Thanks,

Alex

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH 7/7] ALSA: virtio: introduce device suspend/resume support

[Anton Yakovlev]

All running PCM substreams are stopped on device suspend and restarted
on device resume.

Signed-off-by: Anton Yakovlev 
---
 sound/virtio/virtio_card.c| 54 
 sound/virtio/virtio_pcm.c | 40 +++
 sound/virtio/virtio_pcm.h |  6 +++
 sound/virtio/virtio_pcm_ops.c | 93 ---
 4 files changed, 154 insertions(+), 39 deletions(-)

diff --git a/sound/virtio/virtio_card.c b/sound/virtio/virtio_card.c
index 02814c3932ab..2bde324f9b95 100644
--- a/sound/virtio/virtio_card.c
+++ b/sound/virtio/virtio_card.c
@@ -507,6 +507,56 @@ static void virtsnd_config_changed(struct virtio_device 
*vdev)
 "sound device configuration was changed\n");
 }
 
+#ifdef CONFIG_PM_SLEEP
+/**
+ * virtsnd_freeze() - Suspend device.
+ * @vdev: VirtIO parent device.
+ *
+ * Context: Any context that permits to sleep.
+ * Return: 0 on success, -errno on failure.
+ */
+static int virtsnd_freeze(struct virtio_device *vdev)
+{
+   struct virtio_snd *snd = vdev->priv;
+
+   virtsnd_disable_vqs(snd);
+
+   vdev->config->reset(vdev);
+   vdev->config->del_vqs(vdev);
+
+   return 0;
+}
+
+/**
+ * virtsnd_restore() - Resume device.
+ * @vdev: VirtIO parent device.
+ *
+ * Context: Any context that permits to sleep.
+ * Return: 0 on success, -errno on failure.
+ */
+static int virtsnd_restore(struct virtio_device *vdev)
+{
+   struct virtio_snd *snd = vdev->priv;
+   int rc;
+
+   rc = virtsnd_find_vqs(snd);
+   if (rc)
+   return rc;
+
+   virtio_device_ready(vdev);
+
+   virtsnd_enable_vqs(snd);
+
+   if (snd->nsubstreams) {
+   rc = virtsnd_pcm_restore(snd);
+   if (rc)
+   return rc;
+   }
+
+   return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
 static const struct virtio_device_id id_table[] = {
{ VIRTIO_ID_SOUND, VIRTIO_DEV_ANY_ID },
{ 0 },
@@ -520,6 +570,10 @@ static struct virtio_driver virtsnd_driver = {
.probe = virtsnd_probe,
.remove = virtsnd_remove,
.config_changed = virtsnd_config_changed,
+#ifdef CONFIG_PM_SLEEP
+   .freeze = virtsnd_freeze,
+   .restore = virtsnd_restore,
+#endif
 };
 
 static int __init init(void)
diff --git a/sound/virtio/virtio_pcm.c b/sound/virtio/virtio_pcm.c
index 6a1ca6b2c3ca..68d9c6dee13a 100644
--- a/sound/virtio/virtio_pcm.c
+++ b/sound/virtio/virtio_pcm.c
@@ -122,6 +122,7 @@ static int virtsnd_pcm_build_hw(struct virtio_pcm_substream 
*substream,
SNDRV_PCM_INFO_BATCH |
SNDRV_PCM_INFO_BLOCK_TRANSFER |
SNDRV_PCM_INFO_INTERLEAVED |
+   SNDRV_PCM_INFO_RESUME |
SNDRV_PCM_INFO_PAUSE;
 
if (!info->channels_min || info->channels_min > info->channels_max) {
@@ -511,6 +512,45 @@ int virtsnd_pcm_build_devs(struct virtio_snd *snd)
return 0;
 }
 
+#ifdef CONFIG_PM_SLEEP
+/**
+ * virtsnd_pcm_restore() - Resume PCM substreams.
+ * @snd: VirtIO sound device.
+ *
+ * Context: Any context that permits to sleep.
+ * Return: 0 on success, -errno on failure.
+ */
+int virtsnd_pcm_restore(struct virtio_snd *snd)
+{
+   unsigned int i;
+
+   for (i = 0; i < snd->nsubstreams; ++i) {
+   struct virtio_pcm_substream *substream = >substreams[i];
+   struct snd_pcm_substream *ksubstream = substream->substream;
+   int rc;
+
+   if (!substream->suspended)
+   continue;
+
+   /*
+* We restart the substream by executing the standard command
+* sequence. The START command will be sent from a subsequent
+* call to the trigger() callback function after the device has
+* been resumed.
+*/
+   rc = ksubstream->ops->hw_params(ksubstream, NULL);
+   if (rc)
+   return rc;
+
+   rc = ksubstream->ops->prepare(ksubstream);
+   if (rc)
+   return rc;
+   }
+
+   return 0;
+}
+#endif /* CONFIG_PM_SLEEP */
+
 /**
  * virtsnd_pcm_event() - Handle the PCM device event notification.
  * @snd: VirtIO sound device.
diff --git a/sound/virtio/virtio_pcm.h b/sound/virtio/virtio_pcm.h
index a326b921b947..23d0fdd57225 100644
--- a/sound/virtio/virtio_pcm.h
+++ b/sound/virtio/virtio_pcm.h
@@ -41,6 +41,7 @@ struct virtio_pcm_msg;
  * @hw_ptr: Substream hardware pointer value in frames [0 ... buffer_size).
  * @xfer_enabled: Data transfer state (0 - off, 1 - on).
  * @xfer_xrun: Data underflow/overflow state (0 - no xrun, 1 - xrun).
+ * @suspended: Kernel ALSA substream is suspended.
  * @msgs: I/O messages.
  * @msg_last_enqueued: Index of the last I/O message added to the virtqueue.
  * @msg_count: Number of pending I/O messages in the virtqueue.
@@ -60,6 +61,7 @@ struct virtio_pcm_substream {
atomic_t hw_ptr;
atomic_t xfer_enabled;
atomic_t 

[PATCH 3/7] ALSA: virtio: add virtio sound driver

[Anton Yakovlev]

Introduce skeleton of the virtio sound driver. The driver implements
the virtio sound device specification, which has become part of the
virtio standard.

Initial initialization of the device, virtqueues and creation of an
empty ALSA sound device. Also, handling DEVICE_NEEDS_RESET device
status.

Signed-off-by: Anton Yakovlev 
---
 MAINTAINERS   |   2 +
 sound/Kconfig |   2 +
 sound/Makefile|   3 +-
 sound/virtio/Kconfig  |  10 +
 sound/virtio/Makefile |   9 +
 sound/virtio/virtio_card.c| 473 ++
 sound/virtio/virtio_card.h|  92 ++
 sound/virtio/virtio_ctl_msg.c | 293 +++
 sound/virtio/virtio_ctl_msg.h | 122 
 sound/virtio/virtio_pcm.c | 536 ++
 sound/virtio/virtio_pcm.h |  89 ++
 11 files changed, 1630 insertions(+), 1 deletion(-)
 create mode 100644 sound/virtio/Kconfig
 create mode 100644 sound/virtio/Makefile
 create mode 100644 sound/virtio/virtio_card.c
 create mode 100644 sound/virtio/virtio_card.h
 create mode 100644 sound/virtio/virtio_ctl_msg.c
 create mode 100644 sound/virtio/virtio_ctl_msg.h
 create mode 100644 sound/virtio/virtio_pcm.c
 create mode 100644 sound/virtio/virtio_pcm.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 6dfd59eafe82..8a0e9f04402f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18939,8 +18939,10 @@ F: include/uapi/linux/virtio_mem.h
 VIRTIO SOUND DRIVER
 M: Anton Yakovlev 
 L: virtualization@lists.linux-foundation.org
+L: alsa-de...@alsa-project.org (moderated for non-subscribers)
 S: Maintained
 F: include/uapi/linux/virtio_snd.h
+F: sound/virtio/*
 
 VIRTUAL BOX GUEST DEVICE DRIVER
 M: Hans de Goede 
diff --git a/sound/Kconfig b/sound/Kconfig
index 36785410fbe1..e56d96d2b11c 100644
--- a/sound/Kconfig
+++ b/sound/Kconfig
@@ -99,6 +99,8 @@ source "sound/synth/Kconfig"
 
 source "sound/xen/Kconfig"
 
+source "sound/virtio/Kconfig"
+
 endif # SND
 
 endif # !UML
diff --git a/sound/Makefile b/sound/Makefile
index 797ecdcd35e2..04ef04b1168f 100644
--- a/sound/Makefile
+++ b/sound/Makefile
@@ -5,7 +5,8 @@
 obj-$(CONFIG_SOUND) += soundcore.o
 obj-$(CONFIG_DMASOUND) += oss/dmasound/
 obj-$(CONFIG_SND) += core/ i2c/ drivers/ isa/ pci/ ppc/ arm/ sh/ synth/ usb/ \
-   firewire/ sparc/ spi/ parisc/ pcmcia/ mips/ soc/ atmel/ hda/ x86/ xen/
+   firewire/ sparc/ spi/ parisc/ pcmcia/ mips/ soc/ atmel/ hda/ x86/ xen/ \
+   virtio/
 obj-$(CONFIG_SND_AOA) += aoa/
 
 # This one must be compilable even if sound is configured out
diff --git a/sound/virtio/Kconfig b/sound/virtio/Kconfig
new file mode 100644
index ..094cba24ee5b
--- /dev/null
+++ b/sound/virtio/Kconfig
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0+
+# Sound card driver for virtio
+
+config SND_VIRTIO
+   tristate "Virtio sound driver"
+   depends on VIRTIO
+   select SND_PCM
+   select SND_JACK
+   help
+  This is the virtual sound driver for virtio. Say Y or M.
diff --git a/sound/virtio/Makefile b/sound/virtio/Makefile
new file mode 100644
index ..69162a545a41
--- /dev/null
+++ b/sound/virtio/Makefile
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0+
+
+obj-$(CONFIG_SND_VIRTIO) += virtio_snd.o
+
+virtio_snd-objs := \
+   virtio_card.o \
+   virtio_ctl_msg.o \
+   virtio_pcm.o
+
diff --git a/sound/virtio/virtio_card.c b/sound/virtio/virtio_card.c
new file mode 100644
index ..293d497f24e7
--- /dev/null
+++ b/sound/virtio/virtio_card.c
@@ -0,0 +1,473 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Sound card driver for virtio
+ * Copyright (C) 2020  OpenSynergy GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see .
+ */
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "virtio_card.h"
+
+int msg_timeout_ms = MSEC_PER_SEC;
+module_param(msg_timeout_ms, int, 0644);
+MODULE_PARM_DESC(msg_timeout_ms, "Message completion timeout in milliseconds");
+
+static int virtsnd_probe(struct virtio_device *vdev);
+static void virtsnd_remove(struct virtio_device *vdev);
+
+/**
+ * virtsnd_event_send() - Add an event to the event queue.
+ * @vqueue: Underlying event virtqueue.
+ * @event: Event.
+ * @notify: Indicates whether or not to send a notification to the device.
+ * @gfp: Kernel flags for memory allocation.
+ *
+ * Context: 

[PATCH 2/7] uapi: virtio_snd: add the sound device header file

[Anton Yakovlev]

The file contains the definitions for the sound device from the OASIS
virtio spec.

Signed-off-by: Anton Yakovlev 
---
 MAINTAINERS |   6 +
 include/uapi/linux/virtio_snd.h | 361 
 2 files changed, 367 insertions(+)
 create mode 100644 include/uapi/linux/virtio_snd.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 00836f6452f0..6dfd59eafe82 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -18936,6 +18936,12 @@ W: https://virtio-mem.gitlab.io/
 F: drivers/virtio/virtio_mem.c
 F: include/uapi/linux/virtio_mem.h
 
+VIRTIO SOUND DRIVER
+M: Anton Yakovlev 
+L: virtualization@lists.linux-foundation.org
+S: Maintained
+F: include/uapi/linux/virtio_snd.h
+
 VIRTUAL BOX GUEST DEVICE DRIVER
 M: Hans de Goede 
 M: Arnd Bergmann 
diff --git a/include/uapi/linux/virtio_snd.h b/include/uapi/linux/virtio_snd.h
new file mode 100644
index ..1ff6310e54d6
--- /dev/null
+++ b/include/uapi/linux/virtio_snd.h
@@ -0,0 +1,361 @@
+/* SPDX-License-Identifier: BSD-3-Clause */
+/*
+ * Copyright (C) 2020  OpenSynergy GmbH
+ *
+ * This header is BSD licensed so anyone can use the definitions to
+ * implement compatible drivers/servers.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *notice, this list of conditions and the following disclaimer in the
+ *documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of OpenSynergy GmbH nor the names of its contributors
+ *may be used to endorse or promote products derived from this software
+ *without specific prior written permission.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
+ * FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL IBM OR
+ * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
+ * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+ * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
+ * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ */
+#ifndef VIRTIO_SND_IF_H
+#define VIRTIO_SND_IF_H
+
+#include 
+
+/***
+ * CONFIGURATION SPACE
+ */
+struct virtio_snd_config {
+   /* # of available physical jacks */
+   __le32 jacks;
+   /* # of available PCM streams */
+   __le32 streams;
+   /* # of available channel maps */
+   __le32 chmaps;
+};
+
+enum {
+   /* device virtqueue indexes */
+   VIRTIO_SND_VQ_CONTROL = 0,
+   VIRTIO_SND_VQ_EVENT,
+   VIRTIO_SND_VQ_TX,
+   VIRTIO_SND_VQ_RX,
+   /* # of device virtqueues */
+   VIRTIO_SND_VQ_MAX
+};
+
+/***
+ * COMMON DEFINITIONS
+ */
+
+/* supported dataflow directions */
+enum {
+   VIRTIO_SND_D_OUTPUT = 0,
+   VIRTIO_SND_D_INPUT
+};
+
+enum {
+   /* jack control request types */
+   VIRTIO_SND_R_JACK_INFO = 1,
+   VIRTIO_SND_R_JACK_REMAP,
+
+   /* PCM control request types */
+   VIRTIO_SND_R_PCM_INFO = 0x0100,
+   VIRTIO_SND_R_PCM_SET_PARAMS,
+   VIRTIO_SND_R_PCM_PREPARE,
+   VIRTIO_SND_R_PCM_RELEASE,
+   VIRTIO_SND_R_PCM_START,
+   VIRTIO_SND_R_PCM_STOP,
+
+   /* channel map control request types */
+   VIRTIO_SND_R_CHMAP_INFO = 0x0200,
+
+   /* jack event types */
+   VIRTIO_SND_EVT_JACK_CONNECTED = 0x1000,
+   VIRTIO_SND_EVT_JACK_DISCONNECTED,
+
+   /* PCM event types */
+   VIRTIO_SND_EVT_PCM_PERIOD_ELAPSED = 0x1100,
+   VIRTIO_SND_EVT_PCM_XRUN,
+
+   /* common status codes */
+   VIRTIO_SND_S_OK = 0x8000,
+   VIRTIO_SND_S_BAD_MSG,
+   VIRTIO_SND_S_NOT_SUPP,
+   VIRTIO_SND_S_IO_ERR
+};
+
+/* common header */
+struct virtio_snd_hdr {
+   __le32 code;
+};
+
+/* event notification */
+struct virtio_snd_event {
+   /* VIRTIO_SND_EVT_XXX */
+   struct virtio_snd_hdr hdr;
+   /* optional event data */
+   __le32 data;
+};
+
+/* common control request to query an item information */
+struct virtio_snd_query_info {
+   /* VIRTIO_SND_R_XXX_INFO */
+   struct virtio_snd_hdr hdr;
+   /* item start identifier */
+   __le32 start_id;
+   /* item count to query */
+   __le32 

[PATCH 5/7] ALSA: virtio: PCM substream operators

[Anton Yakovlev]

Introduce the operators required for the operation of substreams.

Signed-off-by: Anton Yakovlev 
---
 sound/virtio/Makefile |   3 +-
 sound/virtio/virtio_pcm.c |   5 +-
 sound/virtio/virtio_pcm.h |   2 +
 sound/virtio/virtio_pcm_ops.c | 509 ++
 4 files changed, 517 insertions(+), 2 deletions(-)
 create mode 100644 sound/virtio/virtio_pcm_ops.c

diff --git a/sound/virtio/Makefile b/sound/virtio/Makefile
index 626af3cc3ed7..34493226793f 100644
--- a/sound/virtio/Makefile
+++ b/sound/virtio/Makefile
@@ -6,5 +6,6 @@ virtio_snd-objs := \
virtio_card.o \
virtio_ctl_msg.o \
virtio_pcm.o \
-   virtio_pcm_msg.o
+   virtio_pcm_msg.o \
+   virtio_pcm_ops.o
 
diff --git a/sound/virtio/virtio_pcm.c b/sound/virtio/virtio_pcm.c
index 1ab50dcc88c8..6a1ca6b2c3ca 100644
--- a/sound/virtio/virtio_pcm.c
+++ b/sound/virtio/virtio_pcm.c
@@ -121,7 +121,8 @@ static int virtsnd_pcm_build_hw(struct virtio_pcm_substream 
*substream,
SNDRV_PCM_INFO_MMAP_VALID |
SNDRV_PCM_INFO_BATCH |
SNDRV_PCM_INFO_BLOCK_TRANSFER |
-   SNDRV_PCM_INFO_INTERLEAVED;
+   SNDRV_PCM_INFO_INTERLEAVED |
+   SNDRV_PCM_INFO_PAUSE;
 
if (!info->channels_min || info->channels_min > info->channels_max) {
dev_err(>dev,
@@ -503,6 +504,8 @@ int virtsnd_pcm_build_devs(struct virtio_snd *snd)
if (rc)
return rc;
}
+
+   snd_pcm_set_ops(pcm->pcm, i, _pcm_ops);
}
 
return 0;
diff --git a/sound/virtio/virtio_pcm.h b/sound/virtio/virtio_pcm.h
index d011b7e1d18d..fe467bc05d8b 100644
--- a/sound/virtio/virtio_pcm.h
+++ b/sound/virtio/virtio_pcm.h
@@ -90,6 +90,8 @@ struct virtio_pcm {
struct virtio_pcm_stream streams[SNDRV_PCM_STREAM_LAST + 1];
 };
 
+extern const struct snd_pcm_ops virtsnd_pcm_ops;
+
 int virtsnd_pcm_validate(struct virtio_device *vdev);
 
 int virtsnd_pcm_parse_cfg(struct virtio_snd *snd);
diff --git a/sound/virtio/virtio_pcm_ops.c b/sound/virtio/virtio_pcm_ops.c
new file mode 100644
index ..8d26c1144ad6
--- /dev/null
+++ b/sound/virtio/virtio_pcm_ops.c
@@ -0,0 +1,509 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * Sound card driver for virtio
+ * Copyright (C) 2020  OpenSynergy GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see .
+ */
+#include 
+
+#include "virtio_card.h"
+
+/*
+ * Our main concern here is maintaining the correct state of the underlying I/O
+ * virtqueues. Thus, operators are implemented to support all of the following
+ * possible control paths (excluding all trivial ones):
+ *
+ *+-+
+ *| open()  |<--+
+ *+++   |
+ * v|
+ *  +--+--+ |
+ *   +->| hw_params() |<-+  |
+ *   |  +-+  |  |
+ *   | v |  |
+ *   |   +---+   |  |
+ *   |   | prepare() |<---+  |  |
+ *   |   +---+|  |  |
+ *   | v  |  |  |
+ *   |+-+ |  |  |
+ * +---+  | trigger(START/  | |  |  |
+ * | restore() |  | PAUSE_RELEASE/  |<-+  |  |  |
+ * +---+  | RESUME) |  |  |  |  |
+ *   ^+-+  |  |  |  |
+ *   | v   |  |  |  |
+ *   |   +---+ |  |  |  |
+ *   |   | pointer() | |  |  |  |
+ *   |   +---+ |  |  |  |
+ *   | v   |  |  |  |
+ *   |  +-+|  |  |  |
+ * +---+| trigger(STOP/   |+  |  |  |
+ * | freeze()  |<---| PAUSE_PUSH/ |---+  |  |
+ * +---+| SUSPEND)|  |  |
+ *  +-+  |  |
+ * v |  |
+ *

[PATCH 0/7] ALSA: add virtio sound driver

[Anton Yakovlev]

This series implements a driver part of the virtio sound device
specification v8 [1].

The driver supports PCM playback and capture substreams, jack and
channel map controls. A message-based transport is used to write/read
PCM frames to/from a device.

The series is based (and was actually tested) on Linus's master
branch [2], on top of

commit 1e2a199f6ccd ("Merge tag 'spi-fix-v5.11-rc4' of ...")

As a device part was used OpenSynergy proprietary implementation.

Any comments are very welcome.

[1] https://lists.oasis-open.org/archives/virtio-dev/202003/msg00185.html
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git


Anton Yakovlev (7):
  uapi: virtio_ids: add a sound device type ID from OASIS spec
  uapi: virtio_snd: add the sound device header file
  ALSA: virtio: add virtio sound driver
  ALSA: virtio: handling control and I/O messages for the PCM device
  ALSA: virtio: PCM substream operators
  ALSA: virtio: introduce jack support
  ALSA: virtio: introduce device suspend/resume support

 MAINTAINERS |   8 +
 include/uapi/linux/virtio_ids.h |   1 +
 include/uapi/linux/virtio_snd.h | 361 +++
 sound/Kconfig   |   2 +
 sound/Makefile  |   3 +-
 sound/virtio/Kconfig|  10 +
 sound/virtio/Makefile   |  13 +
 sound/virtio/virtio_card.c  | 593 
 sound/virtio/virtio_card.h  | 121 +++
 sound/virtio/virtio_chmap.c | 237 +
 sound/virtio/virtio_ctl_msg.c   | 293 
 sound/virtio/virtio_ctl_msg.h   | 122 +++
 sound/virtio/virtio_jack.c  | 255 ++
 sound/virtio/virtio_pcm.c   | 582 +++
 sound/virtio/virtio_pcm.h   | 132 +++
 sound/virtio/virtio_pcm_msg.c   | 317 +
 sound/virtio/virtio_pcm_ops.c   | 524 
 17 files changed, 3573 insertions(+), 1 deletion(-)
 create mode 100644 include/uapi/linux/virtio_snd.h
 create mode 100644 sound/virtio/Kconfig
 create mode 100644 sound/virtio/Makefile
 create mode 100644 sound/virtio/virtio_card.c
 create mode 100644 sound/virtio/virtio_card.h
 create mode 100644 sound/virtio/virtio_chmap.c
 create mode 100644 sound/virtio/virtio_ctl_msg.c
 create mode 100644 sound/virtio/virtio_ctl_msg.h
 create mode 100644 sound/virtio/virtio_jack.c
 create mode 100644 sound/virtio/virtio_pcm.c
 create mode 100644 sound/virtio/virtio_pcm.h
 create mode 100644 sound/virtio/virtio_pcm_msg.c
 create mode 100644 sound/virtio/virtio_pcm_ops.c

-- 
2.30.0


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH 6/7] ALSA: virtio: introduce jack support

[Anton Yakovlev]

Enumerate all available jacks and create ALSA controls.

At the moment jacks have a simple implementation and can only be used
to receive notifications about a plugged in/out device.

Signed-off-by: Anton Yakovlev 
---
 sound/virtio/Makefile   |   2 +
 sound/virtio/virtio_card.c  |  33 +
 sound/virtio/virtio_card.h  |  20 +++
 sound/virtio/virtio_chmap.c | 237 +
 sound/virtio/virtio_jack.c  | 255 
 sound/virtio/virtio_pcm.h   |   4 +
 6 files changed, 551 insertions(+)
 create mode 100644 sound/virtio/virtio_chmap.c
 create mode 100644 sound/virtio/virtio_jack.c

diff --git a/sound/virtio/Makefile b/sound/virtio/Makefile
index 34493226793f..2742bddb8874 100644
--- a/sound/virtio/Makefile
+++ b/sound/virtio/Makefile
@@ -4,7 +4,9 @@ obj-$(CONFIG_SND_VIRTIO) += virtio_snd.o
 
 virtio_snd-objs := \
virtio_card.o \
+   virtio_chmap.o \
virtio_ctl_msg.o \
+   virtio_jack.o \
virtio_pcm.o \
virtio_pcm_msg.o \
virtio_pcm_ops.o
diff --git a/sound/virtio/virtio_card.c b/sound/virtio/virtio_card.c
index dc703fc662f5..02814c3932ab 100644
--- a/sound/virtio/virtio_card.c
+++ b/sound/virtio/virtio_card.c
@@ -94,6 +94,11 @@ static void virtsnd_event_notify_cb(struct virtqueue *vqueue)
break;
 
switch (le32_to_cpu(event->hdr.code)) {
+   case VIRTIO_SND_EVT_JACK_CONNECTED:
+   case VIRTIO_SND_EVT_JACK_DISCONNECTED: {
+   virtsnd_jack_event(snd, event);
+   break;
+   }
case VIRTIO_SND_EVT_PCM_PERIOD_ELAPSED:
case VIRTIO_SND_EVT_PCM_XRUN: {
virtsnd_pcm_event(snd, event);
@@ -322,16 +327,36 @@ static int virtsnd_build_devs(struct virtio_snd *snd)
strscpy(snd->card->longname, "VirtIO Sound Card",
sizeof(snd->card->longname));
 
+   rc = virtsnd_jack_parse_cfg(snd);
+   if (rc)
+   return rc;
+
rc = virtsnd_pcm_parse_cfg(snd);
if (rc)
return rc;
 
+   rc = virtsnd_chmap_parse_cfg(snd);
+   if (rc)
+   return rc;
+
+   if (snd->njacks) {
+   rc = virtsnd_jack_build_devs(snd);
+   if (rc)
+   return rc;
+   }
+
if (snd->nsubstreams) {
rc = virtsnd_pcm_build_devs(snd);
if (rc)
return rc;
}
 
+   if (snd->nchmaps) {
+   rc = virtsnd_chmap_build_devs(snd);
+   if (rc)
+   return rc;
+   }
+
return snd_card_register(snd->card);
 }
 
@@ -439,14 +464,22 @@ static void virtsnd_remove(struct virtio_device *vdev)
 
if (stream->substreams)
devm_kfree(>dev, stream->substreams);
+   if (stream->chmaps)
+   devm_kfree(>dev, stream->chmaps);
}
 
devm_kfree(>dev, pcm);
}
 
+   if (snd->jacks)
+   devm_kfree(>dev, snd->jacks);
+
if (snd->substreams)
devm_kfree(>dev, snd->substreams);
 
+   if (snd->chmaps)
+   devm_kfree(>dev, snd->chmaps);
+
devm_kfree(>dev, snd);
 
vdev->priv = NULL;
diff --git a/sound/virtio/virtio_card.h b/sound/virtio/virtio_card.h
index b11c09984882..09c6e9ab80ca 100644
--- a/sound/virtio/virtio_card.h
+++ b/sound/virtio/virtio_card.h
@@ -26,6 +26,7 @@
 #include "virtio_ctl_msg.h"
 #include "virtio_pcm.h"
 
+struct virtio_jack;
 struct virtio_pcm_substream;
 
 /**
@@ -47,8 +48,12 @@ struct virtio_snd_queue {
  * @ctl_msgs: Pending control request list.
  * @event_msgs: Device events.
  * @pcm_list: VirtIO PCM device list.
+ * @jacks: VirtIO jacks.
+ * @njacks: Number of jacks.
  * @substreams: VirtIO PCM substreams.
  * @nsubstreams: Number of PCM substreams.
+ * @chmaps: VirtIO channel maps.
+ * @nchmaps: Number of channel maps.
  */
 struct virtio_snd {
struct virtio_device *vdev;
@@ -58,8 +63,12 @@ struct virtio_snd {
struct list_head ctl_msgs;
struct virtio_snd_event *event_msgs;
struct list_head pcm_list;
+   struct virtio_jack *jacks;
+   unsigned int njacks;
struct virtio_pcm_substream *substreams;
unsigned int nsubstreams;
+   struct virtio_snd_chmap_info *chmaps;
+   unsigned int nchmaps;
 };
 
 /* Message completion timeout in milliseconds (module parameter). */
@@ -98,4 +107,15 @@ virtsnd_pcm_queue(struct virtio_pcm_substream *substream)
return virtsnd_rx_queue(substream->snd);
 }
 
+int virtsnd_jack_parse_cfg(struct virtio_snd *snd);
+
+int virtsnd_jack_build_devs(struct virtio_snd *snd);
+
+void virtsnd_jack_event(struct virtio_snd *snd,
+   struct 

[PATCH 1/7] uapi: virtio_ids: add a sound device type ID from OASIS spec

[Anton Yakovlev]

The OASIS virtio spec defines a sound device type ID that is not
present in the header yet.

Signed-off-by: Anton Yakovlev 
---
 include/uapi/linux/virtio_ids.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
index bc1c0621f5ed..029a2e07a7f9 100644
--- a/include/uapi/linux/virtio_ids.h
+++ b/include/uapi/linux/virtio_ids.h
@@ -51,6 +51,7 @@
 #define VIRTIO_ID_PSTORE   22 /* virtio pstore device */
 #define VIRTIO_ID_IOMMU23 /* virtio IOMMU */
 #define VIRTIO_ID_MEM  24 /* virtio mem */
+#define VIRTIO_ID_SOUND25 /* virtio sound */
 #define VIRTIO_ID_FS   26 /* virtio filesystem */
 #define VIRTIO_ID_PMEM 27 /* virtio pmem */
 #define VIRTIO_ID_MAC80211_HWSIM   29 /* virtio mac80211-hwsim */
-- 
2.30.0


___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[PATCH 4/7] ALSA: virtio: handling control and I/O messages for the PCM device

[Anton Yakovlev]

The driver implements a message-based transport for I/O substream
operations. Before the start of the substream, the hardware buffer is
sliced into I/O messages, the number of which is equal to the current
number of periods. The size of each message is equal to the current
size of one period.

I/O messages are organized in an ordered queue. The completion of the
I/O message indicates an expired period (the only exception is the end
of the stream for the capture substream). Upon completion, the message
is automatically re-added to the end of the queue.

When an I/O message is completed, the hw_ptr value is incremented
unconditionally (to ensure that the hw_ptr always correctly reflects
the state of the messages in the virtqueue). Due to its asynchronous
nature, a message can be completed when the runtime structure no longer
exists. For this reason, the values from this structure are cached in
the virtio substream, which are required to calculate the new value of
the hw_ptr.

Signed-off-by: Anton Yakovlev 
---
 sound/virtio/Makefile |   3 +-
 sound/virtio/virtio_card.c|  33 
 sound/virtio/virtio_card.h|   9 +
 sound/virtio/virtio_pcm.c |   3 +
 sound/virtio/virtio_pcm.h |  31 
 sound/virtio/virtio_pcm_msg.c | 317 ++
 6 files changed, 395 insertions(+), 1 deletion(-)
 create mode 100644 sound/virtio/virtio_pcm_msg.c

diff --git a/sound/virtio/Makefile b/sound/virtio/Makefile
index 69162a545a41..626af3cc3ed7 100644
--- a/sound/virtio/Makefile
+++ b/sound/virtio/Makefile
@@ -5,5 +5,6 @@ obj-$(CONFIG_SND_VIRTIO) += virtio_snd.o
 virtio_snd-objs := \
virtio_card.o \
virtio_ctl_msg.o \
-   virtio_pcm.o
+   virtio_pcm.o \
+   virtio_pcm_msg.o
 
diff --git a/sound/virtio/virtio_card.c b/sound/virtio/virtio_card.c
index 293d497f24e7..dc703fc662f5 100644
--- a/sound/virtio/virtio_card.c
+++ b/sound/virtio/virtio_card.c
@@ -145,6 +145,12 @@ static int virtsnd_find_vqs(struct virtio_snd *snd)
callbacks[VIRTIO_SND_VQ_CONTROL] = virtsnd_ctl_notify_cb;
callbacks[VIRTIO_SND_VQ_EVENT] = virtsnd_event_notify_cb;
 
+   virtio_cread(vdev, struct virtio_snd_config, streams, );
+   if (n) {
+   callbacks[VIRTIO_SND_VQ_TX] = virtsnd_pcm_tx_notify_cb;
+   callbacks[VIRTIO_SND_VQ_RX] = virtsnd_pcm_rx_notify_cb;
+   }
+
rc = virtio_find_vqs(vdev, VIRTIO_SND_VQ_MAX, vqs, callbacks, names,
 NULL);
if (rc) {
@@ -186,15 +192,42 @@ static int virtsnd_find_vqs(struct virtio_snd *snd)
  * virtsnd_enable_vqs() - Enable the event, tx and rx virtqueues.
  * @snd: VirtIO sound device.
  *
+ * The tx queue is enabled only if the device supports playback stream(s).
+ *
+ * The rx queue is enabled only if the device supports capture stream(s).
+ *
  * Context: Any context.
  */
 static void virtsnd_enable_vqs(struct virtio_snd *snd)
 {
+   struct virtio_device *vdev = snd->vdev;
struct virtqueue *vqueue;
+   struct virtio_pcm *pcm;
+   unsigned int npbs = 0;
+   unsigned int ncps = 0;
 
vqueue = snd->queues[VIRTIO_SND_VQ_EVENT].vqueue;
if (!virtqueue_enable_cb(vqueue))
virtsnd_event_notify_cb(vqueue);
+
+   list_for_each_entry(pcm, >pcm_list, list) {
+   npbs += pcm->streams[SNDRV_PCM_STREAM_PLAYBACK].nsubstreams;
+   ncps += pcm->streams[SNDRV_PCM_STREAM_CAPTURE].nsubstreams;
+   }
+
+   if (npbs) {
+   vqueue = snd->queues[VIRTIO_SND_VQ_TX].vqueue;
+   if (!virtqueue_enable_cb(vqueue))
+   dev_warn(>dev,
+"suspicious notification in the TX queue\n");
+   }
+
+   if (ncps) {
+   vqueue = snd->queues[VIRTIO_SND_VQ_RX].vqueue;
+   if (!virtqueue_enable_cb(vqueue))
+   dev_warn(>dev,
+"suspicious notification in the RX queue\n");
+   }
 }
 
 /**
diff --git a/sound/virtio/virtio_card.h b/sound/virtio/virtio_card.h
index be6651a6aaf8..b11c09984882 100644
--- a/sound/virtio/virtio_card.h
+++ b/sound/virtio/virtio_card.h
@@ -89,4 +89,13 @@ virtsnd_rx_queue(struct virtio_snd *snd)
return >queues[VIRTIO_SND_VQ_RX];
 }
 
+static inline struct virtio_snd_queue *
+virtsnd_pcm_queue(struct virtio_pcm_substream *substream)
+{
+   if (substream->direction == SNDRV_PCM_STREAM_PLAYBACK)
+   return virtsnd_tx_queue(substream->snd);
+   else
+   return virtsnd_rx_queue(substream->snd);
+}
+
 #endif /* VIRTIO_SND_CARD_H */
diff --git a/sound/virtio/virtio_pcm.c b/sound/virtio/virtio_pcm.c
index 036990b7b78a..1ab50dcc88c8 100644
--- a/sound/virtio/virtio_pcm.c
+++ b/sound/virtio/virtio_pcm.c
@@ -376,6 +376,7 @@ int virtsnd_pcm_parse_cfg(struct virtio_snd *snd)
 
substream->snd = snd;
substream->sid = i;
+   init_waitqueue_head(>msg_empty);
 
   

RE: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Kasireddy, Vivek]

Hi Alex,

> -Original Message-
> From: Alex Williamson 
> Sent: Tuesday, January 19, 2021 7:40 AM
> To: Kasireddy, Vivek 
> Cc: virtualization@lists.linux-foundation.org; Kim, Dongwon 
> 
> Subject: Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf
> 
> On Tue, 19 Jan 2021 00:28:12 -0800
> Vivek Kasireddy  wrote:
> 
> > Getting a copy of the KVM instance is necessary for mapping Guest
> > pages in the Host.
> >
> > TODO: Instead of invoking the symbol directly, there needs to be a
> > better way of getting a copy of the KVM instance probably by using
> > other notifiers. However, currently, KVM shares its instance only
> > with VFIO and therefore we are compelled to bind the passthrough'd
> > device to vfio-pci.
> 
> Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
> call out to vhost to share a kvm pointer.  I'd prefer to get rid of
> vfio having any knowledge or visibility of the kvm pointer.  Thanks,

[Kasireddy, Vivek] I agree that this is definitely not ideal as I recognize it
in the TODO. However, it looks like VFIO also gets a copy of the KVM 
pointer in a similar manner:

virt/kvm/vfio.c

static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
{
void (*fn)(struct vfio_group *, struct kvm *);

fn = symbol_get(vfio_group_set_kvm);
if (!fn)
return;

fn(group, kvm);

symbol_put(vfio_group_set_kvm);
}

With this patch, I am not suggesting that this is a precedent that should be 
followed 
but it appears there doesn't seem to be an alternative way of getting a copy of 
the KVM 
pointer that is clean and elegant -- unless I have not looked hard enough. I 
guess we
could create a notifier chain with callbacks for VFIO and Vhost that KVM would 
call 
but this would mean modifying KVM.

Also, if I understand correctly, if VFIO does not want to share the KVM pointer 
with
VFIO groups, then I think it would break stuff like mdev which counts on it. 

Thanks,
Vivek

> Alex
> 
> > Signed-off-by: Vivek Kasireddy 
> > ---
> >  drivers/vfio/vfio.c | 9 +
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 4ad8a35667a7..9fb11b1ad3cd 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -2213,11 +2213,20 @@ static int vfio_unregister_iommu_notifier(struct 
> > vfio_group
> *group,
> > return ret;
> >  }
> >
> > +extern void vhost_vdmabuf_get_kvm(unsigned long action, void *data);
> >  void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
> >  {
> > +   void (*fn)(unsigned long, void *);
> > +
> > group->kvm = kvm;
> > blocking_notifier_call_chain(>notifier,
> > VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> > +
> > +   fn = symbol_get(vhost_vdmabuf_get_kvm);
> > +   if (fn) {
> > +   fn(VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> > +   symbol_put(vhost_vdmabuf_get_kvm);
> > +   }
> >  }
> >  EXPORT_SYMBOL_GPL(vfio_group_set_kvm);
> >

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v3 08/11] vduse: Introduce VDUSE - vDPA Device in Userspace

[Randy Dunlap]

Hi,

Documentation comments only:

On 1/18/21 9:07 PM, Xie Yongji wrote:
> 
> Signed-off-by: Xie Yongji 
> ---
>  Documentation/driver-api/vduse.rst |   85 ++
> 
> diff --git a/Documentation/driver-api/vduse.rst 
> b/Documentation/driver-api/vduse.rst
> new file mode 100644
> index ..9418a7f6646b
> --- /dev/null
> +++ b/Documentation/driver-api/vduse.rst
> @@ -0,0 +1,85 @@
> +==
> +VDUSE - "vDPA Device in Userspace"
> +==
> +
> +vDPA (virtio data path acceleration) device is a device that uses a
> +datapath which complies with the virtio specifications with vendor
> +specific control path. vDPA devices can be both physically located on
> +the hardware or emulated by software. VDUSE is a framework that makes it
> +possible to implement software-emulated vDPA devices in userspace.
> +
> +How VDUSE works
> +
> +Each userspace vDPA device is created by the VDUSE_CREATE_DEV ioctl on
> +the VDUSE character device (/dev/vduse). Then a file descriptor pointing
> +to the new resources will be returned, which can be used to implement the
> +userspace vDPA device's control path and data path.
> +
> +To implement control path, the read/write operations to the file descriptor
> +will be used to receive/reply the control messages from/to VDUSE driver.
> +Those control messages are mostly based on the vdpa_config_ops which defines
> +a unified interface to control different types of vDPA device.
> +
> +The following types of messages are provided by the VDUSE framework now:
> +
> +- VDUSE_SET_VQ_ADDR: Set the addresses of the different aspects of virtqueue.
> +
> +- VDUSE_SET_VQ_NUM: Set the size of virtqueue
> +
> +- VDUSE_SET_VQ_READY: Set ready status of virtqueue
> +
> +- VDUSE_GET_VQ_READY: Get ready status of virtqueue
> +
> +- VDUSE_SET_VQ_STATE: Set the state (last_avail_idx) for virtqueue
> +
> +- VDUSE_GET_VQ_STATE: Get the state (last_avail_idx) for virtqueue
> +
> +- VDUSE_SET_FEATURES: Set virtio features supported by the driver
> +
> +- VDUSE_GET_FEATURES: Get virtio features supported by the device
> +
> +- VDUSE_SET_STATUS: Set the device status
> +
> +- VDUSE_GET_STATUS: Get the device status
> +
> +- VDUSE_SET_CONFIG: Write to device specific configuration space
> +
> +- VDUSE_GET_CONFIG: Read from device specific configuration space
> +
> +- VDUSE_UPDATE_IOTLB: Notify userspace to update the memory mapping in 
> device IOTLB
> +
> +Please see include/linux/vdpa.h for details.
> +
> +In the data path, vDPA device's iova regions will be mapped into userspace 
> with
> +the help of VDUSE_IOTLB_GET_FD ioctl on the userspace vDPA device fd:
> +
> +- VDUSE_IOTLB_GET_FD: get the file descriptor to iova region. Userspace can
> +  access this iova region by passing the fd to mmap(2).
> +
> +Besides, the eventfd mechanism is used to trigger interrupt callbacks and
> +receive virtqueue kicks in userspace. The following ioctls on the userspace
> +vDPA device fd are provided to support that:
> +
> +- VDUSE_VQ_SETUP_KICKFD: set the kickfd for virtqueue, this eventfd is used
> +  by VDUSE driver to notify userspace to consume the vring.
> +
> +- VDUSE_VQ_SETUP_IRQFD: set the irqfd for virtqueue, this eventfd is used
> +  by userspace to notify VDUSE driver to trigger interrupt callbacks.
> +
> +MMU-based IOMMU Driver
> +--
> +In virtio-vdpa case, VDUSE framework implements a MMU-based on-chip IOMMU

   an MMU-based

> +driver to support mapping the kernel dma buffer into the userspace iova

DMA

> +region dynamically.
> +
> +The basic idea behind this driver is treating MMU (VA->PA) as IOMMU 
> (IOVA->PA).
> +The driver will set up MMU mapping instead of IOMMU mapping for the DMA 
> transfer
> +so that the userspace process is able to use its virtual address to access
> +the dma buffer in kernel.

   DMA

> +
> +And to avoid security issue, a bounce-buffering mechanism is introduced to
> +prevent userspace accessing the original buffer directly which may contain 
> other
> +kernel data. During the mapping, unmapping, the driver will copy the data 
> from
> +the original buffer to the bounce buffer and back, depending on the 
> direction of
> +the transfer. And the bounce-buffer addresses will be mapped into the user 
> address
> +space instead of the original one.


thanks.
-- 
~Randy

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Alex Williamson]

On Tue, 19 Jan 2021 00:28:12 -0800
Vivek Kasireddy  wrote:

> Getting a copy of the KVM instance is necessary for mapping Guest
> pages in the Host.
> 
> TODO: Instead of invoking the symbol directly, there needs to be a
> better way of getting a copy of the KVM instance probably by using
> other notifiers. However, currently, KVM shares its instance only
> with VFIO and therefore we are compelled to bind the passthrough'd
> device to vfio-pci.

Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
call out to vhost to share a kvm pointer.  I'd prefer to get rid of
vfio having any knowledge or visibility of the kvm pointer.  Thanks,

Alex
 
> Signed-off-by: Vivek Kasireddy 
> ---
>  drivers/vfio/vfio.c | 9 +
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 4ad8a35667a7..9fb11b1ad3cd 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -2213,11 +2213,20 @@ static int vfio_unregister_iommu_notifier(struct 
> vfio_group *group,
>   return ret;
>  }
>  
> +extern void vhost_vdmabuf_get_kvm(unsigned long action, void *data);
>  void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
>  {
> + void (*fn)(unsigned long, void *);
> +
>   group->kvm = kvm;
>   blocking_notifier_call_chain(>notifier,
>   VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> +
> + fn = symbol_get(vhost_vdmabuf_get_kvm);
> + if (fn) {
> + fn(VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> + symbol_put(vhost_vdmabuf_get_kvm);
> + }
>  }
>  EXPORT_SYMBOL_GPL(vfio_group_set_kvm);
>  

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC v3 08/11] vduse: Introduce VDUSE - vDPA Device in Userspace

[Jonathan Corbet]

On Tue, 19 Jan 2021 13:07:53 +0800
Xie Yongji  wrote:

> diff --git a/Documentation/driver-api/vduse.rst 
> b/Documentation/driver-api/vduse.rst
> new file mode 100644
> index ..9418a7f6646b
> --- /dev/null
> +++ b/Documentation/driver-api/vduse.rst
> @@ -0,0 +1,85 @@
> +==
> +VDUSE - "vDPA Device in Userspace"
> +==

Thanks for documenting this feature!  You will, though, need to add this
new document to Documentation/driver-api/index.rst for it to be included
in the docs build.

That said, this would appear to be documentation for user space, right?
So the userspace-api manual is probably a more appropriate place for it.

Thanks,

jon
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v3 06/15] x86/paravirt: switch time pvops functions to use static_call()

[Jürgen Groß via Virtualization]

On 17.12.20 18:31, Michael Kelley wrote:

From: Juergen Gross  Sent: Thursday, December 17, 2020 1:31 AM


The time pvops functions are the only ones left which might be
used in 32-bit mode and which return a 64-bit value.

Switch them to use the static_call() mechanism instead of pvops, as
this allows quite some simplification of the pvops implementation.

Due to include hell this requires to split out the time interfaces
into a new header file.

Signed-off-by: Juergen Gross 
---
  arch/x86/Kconfig  |  1 +
  arch/x86/include/asm/mshyperv.h   | 11 
  arch/x86/include/asm/paravirt.h   | 14 --
  arch/x86/include/asm/paravirt_time.h  | 38 +++
  arch/x86/include/asm/paravirt_types.h |  6 -
  arch/x86/kernel/cpu/vmware.c  |  5 ++--
  arch/x86/kernel/kvm.c |  3 ++-
  arch/x86/kernel/kvmclock.c|  3 ++-
  arch/x86/kernel/paravirt.c| 16 ---
  arch/x86/kernel/tsc.c |  3 ++-
  arch/x86/xen/time.c   | 12 -
  drivers/clocksource/hyperv_timer.c|  5 ++--
  drivers/xen/time.c|  3 ++-
  kernel/sched/sched.h  |  1 +
  14 files changed, 71 insertions(+), 50 deletions(-)
  create mode 100644 arch/x86/include/asm/paravirt_time.h



[snip]
  

diff --git a/arch/x86/include/asm/mshyperv.h b/arch/x86/include/asm/mshyperv.h
index ffc289992d1b..45942d420626 100644
--- a/arch/x86/include/asm/mshyperv.h
+++ b/arch/x86/include/asm/mshyperv.h
@@ -56,17 +56,6 @@ typedef int (*hyperv_fill_flush_list_func)(
  #define hv_get_raw_timer() rdtsc_ordered()
  #define hv_get_vector() HYPERVISOR_CALLBACK_VECTOR

-/*
- * Reference to pv_ops must be inline so objtool
- * detection of noinstr violations can work correctly.
- */
-static __always_inline void hv_setup_sched_clock(void *sched_clock)
-{
-#ifdef CONFIG_PARAVIRT
-   pv_ops.time.sched_clock = sched_clock;
-#endif
-}
-
  void hyperv_vector_handler(struct pt_regs *regs);

  static inline void hv_enable_stimer0_percpu_irq(int irq) {}


[snip]


diff --git a/drivers/clocksource/hyperv_timer.c 
b/drivers/clocksource/hyperv_timer.c
index ba04cb381cd3..1ed79993fc50 100644
--- a/drivers/clocksource/hyperv_timer.c
+++ b/drivers/clocksource/hyperv_timer.c
@@ -21,6 +21,7 @@
  #include 
  #include 
  #include 
+#include 

  static struct clock_event_device __percpu *hv_clock_event;
  static u64 hv_sched_clock_offset __ro_after_init;
@@ -445,7 +446,7 @@ static bool __init hv_init_tsc_clocksource(void)
clocksource_register_hz(_cs_tsc, NSEC_PER_SEC/100);

hv_sched_clock_offset = hv_read_reference_counter();
-   hv_setup_sched_clock(read_hv_sched_clock_tsc);
+   paravirt_set_sched_clock(read_hv_sched_clock_tsc);

return true;
  }
@@ -470,6 +471,6 @@ void __init hv_init_clocksource(void)
clocksource_register_hz(_cs_msr, NSEC_PER_SEC/100);

hv_sched_clock_offset = hv_read_reference_counter();
-   hv_setup_sched_clock(read_hv_sched_clock_msr);
+   static_call_update(pv_sched_clock, read_hv_sched_clock_msr);
  }
  EXPORT_SYMBOL_GPL(hv_init_clocksource);


These Hyper-V changes are problematic as we want to keep hyperv_timer.c
architecture independent.  While only the code for x86/x64 is currently
accepted upstream, code for ARM64 support is in progress.   So we need
to use hv_setup_sched_clock() in hyperv_timer.c, and have the per-arch
implementation in mshyperv.h.


Okay, will switch back to old setup.


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v3 06/15] x86/paravirt: switch time pvops functions to use static_call()

[Jürgen Groß via Virtualization]

On 06.01.21 11:03, Borislav Petkov wrote:

On Thu, Dec 17, 2020 at 10:31:24AM +0100, Juergen Gross wrote:

The time pvops functions are the only ones left which might be
used in 32-bit mode and which return a 64-bit value.

Switch them to use the static_call() mechanism instead of pvops, as
this allows quite some simplification of the pvops implementation.

Due to include hell this requires to split out the time interfaces
into a new header file.


I guess you can add Peter's patch to your set, no?

https://lkml.kernel.org/r/20201110005609.40989-3-frede...@kernel.org


With a slight modification, yes. :-)


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: application/pgp-keys


OpenPGP_signature
Description: OpenPGP digital signature
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH linux-next v3 6/6] vdpa_sim_net: Add support for user supported devices

[Jason Wang]

On 2021/1/15 下午2:27, Parav Pandit wrote:

With that mac, mtu as optional input fields provide the necessary flexibility

for different stacks to take appropriate shape as they desire.


Thanks for the clarification. I think we'd better document the above in the
patch that introduces the mac setting from management API.

Yes. Will do.
Thanks.



Adding Sean.

Regarding to mac address setting. Do we plan to allow to modify mac 
address after the creation? It looks like Openstack wants this.


Sean may share more information on this.

Thanks

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH net-next v7] vhost_net: avoid tx queue stuck when sendmsg fails

[Michael S. Tsirkin]

On Mon, Jan 18, 2021 at 02:33:29PM -0800, Jakub Kicinski wrote:
> On Fri, 15 Jan 2021 12:46:20 +0800 wangyunjian wrote:
> > From: Yunjian Wang 
> > 
> > Currently the driver doesn't drop a packet which can't be sent by tun
> > (e.g bad packet). In this case, the driver will always process the
> > same packet lead to the tx queue stuck.
> > 
> > To fix this issue:
> > 1. in the case of persistent failure (e.g bad packet), the driver
> >can skip this descriptor by ignoring the error.
> > 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
> >the driver schedules the worker to try again.
> > 
> > Signed-off-by: Yunjian Wang 
> 
> Michael, LMK if you want to have a closer look otherwise I'll apply
> tomorrow.

Thanks for the reminder. Acked.

-- 
MST

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH net-next v7] vhost_net: avoid tx queue stuck when sendmsg fails

[Michael S. Tsirkin]

On Fri, Jan 15, 2021 at 12:46:20PM +0800, wangyunjian wrote:
> From: Yunjian Wang 
> 
> Currently the driver doesn't drop a packet which can't be sent by tun
> (e.g bad packet). In this case, the driver will always process the
> same packet lead to the tx queue stuck.
> 
> To fix this issue:
> 1. in the case of persistent failure (e.g bad packet), the driver
>can skip this descriptor by ignoring the error.
> 2. in the case of transient failure (e.g -ENOBUFS, -EAGAIN and -ENOMEM),
>the driver schedules the worker to try again.
> 
> Signed-off-by: Yunjian Wang 

Acked-by: Michael S. Tsirkin 

> ---
> v7:
>* code rebase
> v6:
>* update code styles and commit log
> ---
>  drivers/vhost/net.c | 26 ++
>  1 file changed, 14 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index 3b744031ec8f..df82b124170e 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -828,14 +828,15 @@ static void handle_tx_copy(struct vhost_net *net, 
> struct socket *sock)
>   msg.msg_flags &= ~MSG_MORE;
>   }
>  
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
>   err = sock->ops->sendmsg(sock, , len);
>   if (unlikely(err < 0)) {
> - vhost_discard_vq_desc(vq, 1);
> - vhost_net_enable_vq(net, vq);
> - break;
> - }
> - if (err != len)
> + if (err == -EAGAIN || err == -ENOMEM || err == 
> -ENOBUFS) {
> + vhost_discard_vq_desc(vq, 1);
> + vhost_net_enable_vq(net, vq);
> + break;
> + }
> + pr_debug("Fail to send packet: err %d", err);
> + } else if (unlikely(err != len))
>   pr_debug("Truncated TX packet: len %d != %zd\n",
>err, len);
>  done:
> @@ -924,7 +925,6 @@ static void handle_tx_zerocopy(struct vhost_net *net, 
> struct socket *sock)
>   msg.msg_flags &= ~MSG_MORE;
>   }
>  
> - /* TODO: Check specific error and bomb out unless ENOBUFS? */
>   err = sock->ops->sendmsg(sock, , len);
>   if (unlikely(err < 0)) {
>   if (zcopy_used) {
> @@ -933,11 +933,13 @@ static void handle_tx_zerocopy(struct vhost_net *net, 
> struct socket *sock)
>   nvq->upend_idx = ((unsigned)nvq->upend_idx - 1)
>   % UIO_MAXIOV;
>   }
> - vhost_discard_vq_desc(vq, 1);
> - vhost_net_enable_vq(net, vq);
> - break;
> - }
> - if (err != len)
> + if (err == -EAGAIN || err == -ENOMEM || err == 
> -ENOBUFS) {
> + vhost_discard_vq_desc(vq, 1);
> + vhost_net_enable_vq(net, vq);
> + break;
> + }
> + pr_debug("Fail to send packet: err %d", err);
> + } else if (unlikely(err != len))
>   pr_debug("Truncated TX packet: "
>" len %d != %zd\n", err, len);
>   if (!zcopy_used)
> -- 
> 2.23.0

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH bpf-next v2 0/3] xsk: build skb by page

[Michael S. Tsirkin]

On Tue, Jan 19, 2021 at 05:45:09PM +0800, Xuan Zhuo wrote:
> v2:
> 1. add priv_flags IFF_TX_SKB_NO_LINEAR instead of netdev_feature
> 2. split the patch to three:
> a. add priv_flags IFF_TX_SKB_NO_LINEAR
> b. virtio net add priv_flags IFF_TX_SKB_NO_LINEAR
> c. When there is support this flag, construct skb without linear space
> 3. use ERR_PTR() and PTR_ERR() to handle the err
> 
> 
> v1 message log:
> ---
> 
> This patch is used to construct skb based on page to save memory copy
> overhead.
> 
> This has one problem:
> 
> We construct the skb by fill the data page as a frag into the skb. In
> this way, the linear space is empty, and the header information is also
> in the frag, not in the linear space, which is not allowed for some
> network cards. For example, Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx] will get the following error message:
> 
> mlx5_core :3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, 
> opcode 0xd, syndrome 0x1, vendor syndrome 0x68
> : 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2
> WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64
> : 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00
> 0010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00
> 0030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> mlx5_core :3b:00.1 eth1: ERR CQE on SQ: 0x1dbb
> 
> I also tried to use build_skb to construct skb, but because of the
> existence of skb_shinfo, it must be behind the linear space, so this
> method is not working. We can't put skb_shinfo on desc->addr, it will be
> exposed to users, this is not safe.
> 
> Finally, I added a feature NETIF_F_SKB_NO_LINEAR to identify whether the
> network card supports the header information of the packet in the frag
> and not in the linear space.
> 
>  Performance Testing 
> 
> The test environment is Aliyun ECS server.
> Test cmd:
> ```
> xdpsock -i eth0 -t  -S -s 
> ```
> 
> Test result data:
> 
> size64  512 10241500
> copy1916747 1775988 1600203 1440054
> page1974058 1953655 1945463 1904478
> percent 3.0%10.0%   21.58%  32.3%

Just making sure, are these test results with v2?

> 
> Xuan Zhuo (3):
>   net: add priv_flags for allow tx skb without linear
>   virtio-net: support IFF_TX_SKB_NO_LINEAR
>   xsk: build skb by page
> 
>  drivers/net/virtio_net.c  |   3 +-
>  include/linux/netdevice.h |   3 ++
>  net/xdp/xsk.c | 112 
> ++
>  3 files changed, 99 insertions(+), 19 deletions(-)
> 
> --
> 1.8.3.1

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH RFC v1 00/15] iommu/virtio: Nested stage support with Arm

[Auger Eric]

Hi Vivek,

On 1/15/21 1:13 PM, Vivek Gautam wrote:
> This patch-series aims at enabling Nested stage translation in guests
> using virtio-iommu as the paravirtualized iommu. The backend is supported
> with Arm SMMU-v3 that provides nested stage-1 and stage-2 translation.
> 
> This series derives its purpose from various efforts happening to add
> support for Shared Virtual Addressing (SVA) in host and guest. On Arm,
> most of the support for SVA has already landed. The support for nested
> stage translation and fault reporting to guest has been proposed [1].
> The related changes required in VFIO [2] framework have also been put
> forward.
> 
> This series proposes changes in virtio-iommu to program PASID tables
> and related stage-1 page tables. A simple iommu-pasid-table library
> is added for this purpose that interacts with vendor drivers to
> allocate and populate PASID tables.
> In Arm SMMUv3 we propose to pull the Context Descriptor (CD) management
> code out of the arm-smmu-v3 driver and add that as a glue vendor layer
> to support allocating CD tables, and populating them with right values.
> These CD tables are essentially the PASID tables and contain stage-1
> page table configurations too.
> A request to setup these CD tables come from virtio-iommu driver using
> the iommu-pasid-table library when running on Arm. The virtio-iommu
> then pass these PASID tables to the host using the right virtio backend
> and support in VMM.
> 
> For testing we have added necessary support in kvmtool. The changes in
> kvmtool are based on virtio-iommu development branch by Jean-Philippe
> Brucker [3].
> 
> The tested kernel branch contains following in the order bottom to top
> on the git hash -
> a) v5.11-rc3
> b) arm-smmu-v3 [1] and vfio [2] changes from Eric to add nested page
>table support for Arm.
> c) Smmu test engine patches from Jean-Philippe's branch [4]
> d) This series
> e) Domain nesting info patches [5][6][7].
> f) Changes to add arm-smmu-v3 specific nesting info (to be sent to
>the list).
> 
> This kernel is tested on Neoverse reference software stack with
> Fixed virtual platform. Public version of the software stack and
> FVP is available here[8][9].
> 
> A big thanks to Jean-Philippe for his contributions towards this work
> and for his valuable guidance.
> 
> [1] 
> https://lore.kernel.org/linux-iommu/20201118112151.25412-1-eric.au...@redhat.com/T/
> [2] 
> https://lore.kernel.org/kvmarm/20201116110030.32335-12-eric.au...@redhat.com/T/
> [3] https://jpbrucker.net/git/kvmtool/log/?h=virtio-iommu/devel
> [4] https://jpbrucker.net/git/linux/log/?h=sva/smmute
> [5] 
> https://lore.kernel.org/kvm/1599734733-6431-2-git-send-email-yi.l@intel.com/
> [6] 
> https://lore.kernel.org/kvm/1599734733-6431-3-git-send-email-yi.l@intel.com/
> [7] 
> https://lore.kernel.org/kvm/1599734733-6431-4-git-send-email-yi.l@intel.com/
> [8] 
> https://developer.arm.com/tools-and-software/open-source-software/arm-platforms-software/arm-ecosystem-fvps
> [9] 
> https://git.linaro.org/landing-teams/working/arm/arm-reference-platforms.git/about/docs/rdn1edge/user-guide.rst

Could you share a public branch where we could find all the kernel pieces.

Thank you in advance

Best Regards

Eric
> 
> Jean-Philippe Brucker (6):
>   iommu/virtio: Add headers for table format probing
>   iommu/virtio: Add table format probing
>   iommu/virtio: Add headers for binding pasid table in iommu
>   iommu/virtio: Add support for INVALIDATE request
>   iommu/virtio: Attach Arm PASID tables when available
>   iommu/virtio: Add support for Arm LPAE page table format
> 
> Vivek Gautam (9):
>   iommu/arm-smmu-v3: Create a Context Descriptor library
>   iommu: Add a simple PASID table library
>   iommu/arm-smmu-v3: Update drivers to work with iommu-pasid-table
>   iommu/arm-smmu-v3: Update CD base address info for user-space
>   iommu/arm-smmu-v3: Set sync op from consumer driver of cd-lib
>   iommu: Add asid_bits to arm smmu-v3 stage1 table info
>   iommu/virtio: Update table format probing header
>   iommu/virtio: Prepare to add attach pasid table infrastructure
>   iommu/virtio: Update fault type and reason info for viommu fault
> 
>  drivers/iommu/arm/arm-smmu-v3/Makefile|   2 +-
>  .../arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c  | 283 +++
>  .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  16 +-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 268 +--
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h   |   4 +-
>  drivers/iommu/iommu-pasid-table.h | 140 
>  drivers/iommu/virtio-iommu.c  | 692 +-
>  include/uapi/linux/iommu.h|   2 +-
>  include/uapi/linux/virtio_iommu.h | 158 +++-
>  9 files changed, 1303 insertions(+), 262 deletions(-)
>  create mode 100644 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-cd-lib.c
>  create mode 100644 drivers/iommu/iommu-pasid-table.h
> 

___
Virtualization 

[RFC 1/3] virtio: Introduce Vdmabuf driver

[Vivek Kasireddy]

This driver "transfers" a dmabuf created on the Guest to the Host.
A common use-case for such a transfer includes sharing the scanout
buffer created by a display server or a compositor running in the
Guest with Qemu UI -- running on the Host.

The "transfer" is accomplished by sharing the PFNs of all the pages
associated with the dmabuf and having a new dmabuf created on the
Host that is backed up by the pages mapped from the Guest.

Signed-off-by: Dongwon Kim 
Signed-off-by: Vivek Kasireddy 
---
 drivers/virtio/Kconfig  |   8 +
 drivers/virtio/Makefile |   1 +
 drivers/virtio/virtio_vdmabuf.c | 973 
 include/linux/virtio_vdmabuf.h  | 271 
 include/uapi/linux/virtio_ids.h |   1 +
 include/uapi/linux/virtio_vdmabuf.h |  99 +++
 6 files changed, 1353 insertions(+)
 create mode 100644 drivers/virtio/virtio_vdmabuf.c
 create mode 100644 include/linux/virtio_vdmabuf.h
 create mode 100644 include/uapi/linux/virtio_vdmabuf.h

diff --git a/drivers/virtio/Kconfig b/drivers/virtio/Kconfig
index 7b41130d3f35..e563c12f711e 100644
--- a/drivers/virtio/Kconfig
+++ b/drivers/virtio/Kconfig
@@ -139,4 +139,12 @@ config VIRTIO_DMA_SHARED_BUFFER
 This option adds a flavor of dma buffers that are backed by
 virtio resources.
 
+config VIRTIO_VDMABUF
+   bool "Enables Vdmabuf driver in guest os"
+   default n
+   depends on VIRTIO
+   help
+This driver provides a way to share the dmabufs created in
+the Guest with the Host.
+
 endif # VIRTIO_MENU
diff --git a/drivers/virtio/Makefile b/drivers/virtio/Makefile
index 591e6f72aa54..b4bb0738009c 100644
--- a/drivers/virtio/Makefile
+++ b/drivers/virtio/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_VIRTIO_INPUT) += virtio_input.o
 obj-$(CONFIG_VIRTIO_VDPA) += virtio_vdpa.o
 obj-$(CONFIG_VIRTIO_MEM) += virtio_mem.o
 obj-$(CONFIG_VIRTIO_DMA_SHARED_BUFFER) += virtio_dma_buf.o
+obj-$(CONFIG_VIRTIO_VDMABUF) += virtio_vdmabuf.o
diff --git a/drivers/virtio/virtio_vdmabuf.c b/drivers/virtio/virtio_vdmabuf.c
new file mode 100644
index ..e377114c2a2b
--- /dev/null
+++ b/drivers/virtio/virtio_vdmabuf.c
@@ -0,0 +1,973 @@
+// SPDX-License-Identifier: (MIT OR GPL-2.0)
+
+/*
+ * Copyright © 2021 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ *Dongwon Kim 
+ *Mateusz Polrola 
+ *Vivek Kasireddy 
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#define VIRTIO_VDMABUF_MAX_ID INT_MAX
+#define REFS_PER_PAGE (PAGE_SIZE/sizeof(long))
+#define NEW_BUF_ID_GEN(vmid, cnt) (((vmid & 0x) << 32) | \
+   ((cnt) & 0x))
+
+/* one global drv object */
+static struct virtio_vdmabuf_info *drv_info;
+
+struct virtio_vdmabuf {
+   /* virtio device structure */
+   struct virtio_device *vdev;
+
+   /* virtual queue array */
+   struct virtqueue *vq;
+
+   /* ID of guest OS */
+   u64 vmid;
+
+   /* spin lock that needs to be acquired before accessing
+* virtual queue
+*/
+   spinlock_t vq_lock;
+   struct mutex rx_lock;
+
+   /* workqueue */
+   struct workqueue_struct *wq;
+   struct work_struct rx_work;
+   struct virtio_vdmabuf_event_queue *evq;
+};
+
+static virtio_vdmabuf_buf_id_t get_buf_id(void)
+{
+   struct virtio_vdmabuf *vdmabuf = drv_info->priv;
+   virtio_vdmabuf_buf_id_t buf_id = {0, {0, 0} };
+   static int count = 0;
+
+   count = count < VIRTIO_VDMABUF_MAX_ID ? count + 1 : 0;
+   buf_id.id = NEW_BUF_ID_GEN(vdmabuf->vmid, count);
+
+   /* random data embedded in the id for security */
+   get_random_bytes(_id.rng_key[0], 8);
+
+   return buf_id;
+}
+
+/* sharing pages for original DMABUF 

[RFC 2/3] vhost: Add Vdmabuf backend

[Vivek Kasireddy]

This backend acts as the counterpart to the Vdmabuf Virtio frontend.
When it receives a new export event from the frontend, it raises an
event to alert the Qemu UI/userspace. Qemu then "imports" this buffer
using the Unique ID.

As part of the import step, a new dmabuf is created on the Host using
the page information obtained from the Guest. The fd associated with
this dmabuf is made available to Qemu UI/userspace which then creates
a texture from it for the purpose of displaying it.

Signed-off-by: Dongwon Kim 
Signed-off-by: Vivek Kasireddy 
---
 drivers/vhost/Kconfig   |9 +
 drivers/vhost/Makefile  |3 +
 drivers/vhost/vdmabuf.c | 1332 +++
 3 files changed, 1344 insertions(+)
 create mode 100644 drivers/vhost/vdmabuf.c

diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig
index 587fbae06182..1f1c51c4499e 100644
--- a/drivers/vhost/Kconfig
+++ b/drivers/vhost/Kconfig
@@ -89,4 +89,13 @@ config VHOST_CROSS_ENDIAN_LEGACY
 
  If unsure, say "N".
 
+config VHOST_VDMABUF
+   bool "Vhost backend for the Vdmabuf driver"
+   depends on EVENTFD
+   select VHOST
+   default n
+   help
+ This driver works in pair with the Virtio Vdmabuf frontend. It can
+ be used to create a dmabuf using the pages shared by the Guest.
+
 endif
diff --git a/drivers/vhost/Makefile b/drivers/vhost/Makefile
index f3e1897cce85..5c2cea4a7eaf 100644
--- a/drivers/vhost/Makefile
+++ b/drivers/vhost/Makefile
@@ -17,3 +17,6 @@ obj-$(CONFIG_VHOST)   += vhost.o
 
 obj-$(CONFIG_VHOST_IOTLB) += vhost_iotlb.o
 vhost_iotlb-y := iotlb.o
+
+obj-$(CONFIG_VHOST_VDMABUF) += vhost_vdmabuf.o
+vhost_vdmabuf-y := vdmabuf.o
diff --git a/drivers/vhost/vdmabuf.c b/drivers/vhost/vdmabuf.c
new file mode 100644
index ..7e2576fc2c0d
--- /dev/null
+++ b/drivers/vhost/vdmabuf.c
@@ -0,0 +1,1332 @@
+// SPDX-License-Identifier: (MIT OR GPL-2.0)
+
+/*
+ * Copyright © 2021 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ *Dongwon Kim 
+ *Mateusz Polrola 
+ *Vivek Kasireddy 
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vhost.h"
+
+#define REFS_PER_PAGE (PAGE_SIZE/sizeof(long))
+
+static struct virtio_vdmabuf_info *drv_info;
+
+struct kvm_instance {
+   struct kvm *kvm;
+   struct list_head link;
+};
+
+struct vhost_vdmabuf {
+   struct vhost_dev dev;
+   struct vhost_virtqueue vq;
+   struct vhost_work tx_work;
+   struct virtio_vdmabuf_event_queue *evq;
+   u64 vmid;
+
+   /* synchronization between transmissions */
+   struct mutex tx_mutex;
+   /* synchronization on tx and rx */
+   struct mutex vq_mutex;
+
+   struct virtio_vdmabuf_txmsg next;
+   struct list_head list;
+   struct kvm *kvm;
+};
+
+static inline void vhost_vdmabuf_add(struct vhost_vdmabuf *new)
+{
+   list_add_tail(>list, _info->head_vdmabuf_list);
+}
+
+static inline struct vhost_vdmabuf *vhost_vdmabuf_find(u64 vmid)
+{
+   struct vhost_vdmabuf *found;
+
+   list_for_each_entry(found, _info->head_vdmabuf_list, list)
+   if (found->vmid == vmid)
+   return found;
+
+   return NULL;
+}
+
+static inline bool vhost_vdmabuf_del(struct vhost_vdmabuf *vdmabuf)
+{
+   struct vhost_vdmabuf *iter, *temp;
+
+   list_for_each_entry_safe(iter, temp,
+_info->head_vdmabuf_list,
+list)
+   if (iter == vdmabuf) {
+   list_del(>list);
+   return true;
+   }
+
+   return false;
+}
+
+static inline void vhost_vdmabuf_del_all(void)
+{
+   struct vhost_vdmabuf *iter, *temp;
+
+   

[RFC 3/3] vfio: Share the KVM instance with Vdmabuf

[Vivek Kasireddy]

Getting a copy of the KVM instance is necessary for mapping Guest
pages in the Host.

TODO: Instead of invoking the symbol directly, there needs to be a
better way of getting a copy of the KVM instance probably by using
other notifiers. However, currently, KVM shares its instance only
with VFIO and therefore we are compelled to bind the passthrough'd
device to vfio-pci.

Signed-off-by: Vivek Kasireddy 
---
 drivers/vfio/vfio.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
index 4ad8a35667a7..9fb11b1ad3cd 100644
--- a/drivers/vfio/vfio.c
+++ b/drivers/vfio/vfio.c
@@ -2213,11 +2213,20 @@ static int vfio_unregister_iommu_notifier(struct 
vfio_group *group,
return ret;
 }
 
+extern void vhost_vdmabuf_get_kvm(unsigned long action, void *data);
 void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
 {
+   void (*fn)(unsigned long, void *);
+
group->kvm = kvm;
blocking_notifier_call_chain(>notifier,
VFIO_GROUP_NOTIFY_SET_KVM, kvm);
+
+   fn = symbol_get(vhost_vdmabuf_get_kvm);
+   if (fn) {
+   fn(VFIO_GROUP_NOTIFY_SET_KVM, kvm);
+   symbol_put(vhost_vdmabuf_get_kvm);
+   }
 }
 EXPORT_SYMBOL_GPL(vfio_group_set_kvm);
 
-- 
2.26.2

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

[RFC 0/3] Introduce Vdmabuf driver

[Vivek Kasireddy]

The Virtual dmabuf or Virtio based dmabuf (Vdmabuf) driver can be used
to "transfer" a page-backed dmabuf created in the Guest to the Host
without making any copies. A use-case where this driver would be a good 
fit is a multi-GPU system (perhaps one discrete and one integrated)
where one of the GPUs does not have access to the display/connectors/outputs.
This could be a embedded system design decision or a restriction made at
the firmware/BIOS level. When such a GPU is passthrough'd to a Guest OS,
this driver can help in transferring the scanout buffer(s) (rendered
using the native rendering stack) to the Host for the purpose of
displaying them.

The userspace component running in the Guest that transfers the dmabuf
is referred to as the producer or exporter and its counterpart running
in the Host is referred to as importer or consumer. For instance, a
Wayland compositor would potentially be a producer and Qemu UI would
be a consumer. It is the producer's responsibility to not reuse or
destroy the shared buffer while it is still being used by the consumer.
The consumer would send a release cmd indicating that it is done after
which the shared buffer can be safely used again by the producer. One
way the producer can prevent accidental re-use of the shared buffer is
to lock the buffer when it exports it and unlock it after it gets a 
release cmd. As an example, the GBM API provides a simple way to lock 
and unlock a surface's buffers.

For each dmabuf that is to be shared with the Host, a 128-bit unique
ID is generated that identifies this buffer across the whole system.
This ID is a combination of the Qemu process ID, a counter and a
randomizer. We could potentially use UUID API but we currently use
the above mentioned combination to identify the source of the buffer
at any given time for bookkeeping.

Vivek Kasireddy (3):
  virtio: Introduce Vdmabuf driver
  vhost: Add Vdmabuf backend
  vfio: Share the KVM instance with Vdmabuf

 drivers/vfio/vfio.c |9 +
 drivers/vhost/Kconfig   |9 +
 drivers/vhost/Makefile  |3 +
 drivers/vhost/vdmabuf.c | 1332 +++
 drivers/virtio/Kconfig  |8 +
 drivers/virtio/Makefile |1 +
 drivers/virtio/virtio_vdmabuf.c |  973 +++
 include/linux/virtio_vdmabuf.h  |  271 ++
 include/uapi/linux/virtio_ids.h |1 +
 include/uapi/linux/virtio_vdmabuf.h |   99 ++
 10 files changed, 2706 insertions(+)
 create mode 100644 drivers/vhost/vdmabuf.c
 create mode 100644 drivers/virtio/virtio_vdmabuf.c
 create mode 100644 include/linux/virtio_vdmabuf.h
 create mode 100644 include/uapi/linux/virtio_vdmabuf.h

-- 
2.26.2

___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization