Re: [RFC 1/2] target: Add documentation on the target userspace pass-through driver

2014-08-30 Thread Richard W.M. Jones
On Tue, Jul 01, 2014 at 12:11:14PM -0700, Andy Grover wrote: Describes the driver and its interface to make it possible for user programs to back a LIO-exported LUN. Signed-off-by: Andy Grover agro...@redhat.com --- Documentation/target/tcmu-design.txt | 210

Re: [RFC 1/2] target: Add documentation on the target userspace pass-through driver

2014-08-31 Thread Richard W.M. Jones
On Sun, Aug 31, 2014 at 12:49:26PM -0700, Andy Grover wrote: Thanks for the feedback. I am undoubtedly too close to the details, because I thought I *was* explaining things :) Yeah, sorry it came across as a bit harsh. Benoit did explain it to me so I understood it in the end (I think!) This

Re: [PATCH] [SCSI] virtio-scsi: Initialize scatterlist structure.

2012-10-01 Thread Richard W.M. Jones
On Mon, Oct 01, 2012 at 03:13:01PM +0200, Paolo Bonzini wrote: Il 20/08/2012 16:05, Paolo Bonzini ha scritto: Il 20/08/2012 16:04, Richard W.M. Jones ha scritto: From: Richard W.M. Jones rjo...@redhat.com The sg struct is used without being initialized. https://bugzilla.redhat.com

[PATCH] [SCSI] virtio-scsi: Initialize scatterlist structure.

2012-08-20 Thread Richard W.M. Jones
From: Richard W.M. Jones rjo...@redhat.com The sg struct is used without being initialized. https://bugzilla.redhat.com/show_bug.cgi?id=847548 Signed-off-by: Richard W.M. Jones rjo...@redhat.com --- drivers/scsi/virtio_scsi.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: Increased memory usage with scsi-mq

2017-08-10 Thread Richard W.M. Jones
OK this is looking a bit better now. With scsi-mq enabled: 175 disks virtqueue_size=64: 318 disks * virtqueue_size=16: 775 disks * With scsi-mq disabled: 1755 disks * = new results I also ran the whole libguestfs test suite with

Re: Increased memory usage with scsi-mq

2017-08-10 Thread Richard W.M. Jones
On Thu, Aug 10, 2017 at 02:53:58PM +0200, Paolo Bonzini wrote: > can_queue and cmd_per_lun are different. can_queue should be set to the > value of vq->vring.num where vq is the command virtqueue (the first one > is okay if there's >1). > > If you want to change it, you'll have to do so in QEMU.

[PATCH 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.

2017-08-10 Thread Richard W.M. Jones
ch command in the queue takes 1 virtqueue entry, so the number of commands which can be queued is equal to the length of the virtqueue. Note I intend to send a patch to qemu to allow the virtqueue size to be configured from the qemu command line. Thanks Paolo Bonzini, Christoph Hellwig. Signed-off-by: R

[PATCH v2 0/2] virtio_scsi: Set can_queue based on size of virtqueue.

2017-08-10 Thread Richard W.M. Jones
v1 was here: https://lkml.org/lkml/2017/8/10/689 v1 -> v2: Remove .can_queue field from the templates. Rich.

Re: [PATCH 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.

2017-08-10 Thread Richard W.M. Jones
On Fri, Aug 11, 2017 at 12:21:16AM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 10, 2017 at 05:40:34PM +0100, Richard W.M. Jones wrote: > > If using indirect descriptors, you can make the total_sg as large as > > you want. > > That would be a spec violation though, even if

Re: [PATCH 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.

2017-08-10 Thread Richard W.M. Jones
On Fri, Aug 11, 2017 at 12:31:44AM +0300, Michael S. Tsirkin wrote: > Then we probably should fail probe if vq size is too small. What does this mean? Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog:

Re: [PATCH 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.

2017-08-10 Thread Richard W.M. Jones
On Fri, Aug 11, 2017 at 12:41:47AM +0300, Michael S. Tsirkin wrote: > On Thu, Aug 10, 2017 at 10:35:11PM +0100, Richard W.M. Jones wrote: > > On Fri, Aug 11, 2017 at 12:31:44AM +0300, Michael S. Tsirkin wrote: > > > Then we probably should fail probe if vq size is too small.

[PATCH 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.

2017-08-10 Thread Richard W.M. Jones
If using indirect descriptors, you can make the total_sg as large as you want. If not, BUG is too serious because the function later returns -ENOSPC. Thanks Paolo Bonzini, Christoph Hellwig. Signed-off-by: Richard W.M. Jones <rjo...@redhat.com> --- drivers/virtio/virtio_ring.c | 5 ++

[PATCH 0/2] virtio_scsi: Set can_queue based on size of virtqueue.

2017-08-10 Thread Richard W.M. Jones
Earlier discussion: https://lkml.org/lkml/2017/8/4/601 "Increased memory usage with scsi-mq" Downstream bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1478201

[PATCH v2 2/2] virtio: virtio_scsi: Set can_queue to the length of the virtqueue.

2017-08-10 Thread Richard W.M. Jones
ch command in the queue takes 1 virtqueue entry, so the number of commands which can be queued is equal to the length of the virtqueue. Note I intend to send a patch to qemu to allow the virtqueue size to be configured from the qemu command line. Thanks Paolo Bonzini, Christoph Hellwig. Signed-off-by: R

[PATCH v2 1/2] virtio: Reduce BUG if total_sg > virtqueue size to WARN.

2017-08-10 Thread Richard W.M. Jones
If using indirect descriptors, you can make the total_sg as large as you want. If not, BUG is too serious because the function later returns -ENOSPC. Thanks Paolo Bonzini, Christoph Hellwig. Signed-off-by: Richard W.M. Jones <rjo...@redhat.com> Reviewed-by: Paolo Bonzini <pbonz...@r

Re: Increased memory usage with scsi-mq

2017-08-10 Thread Richard W.M. Jones
23e49846189b2f55f3f02b70a290d4be237ed5 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" <rjo...@redhat.com> Date: Thu, 10 Aug 2017 12:21:47 +0100 Subject: [PATCH] scsi: virtio_scsi: Set can_queue based on cmd_per_lun passed by hypervisor. Signed-off-by: Richard W.M. Jones

Increased memory usage with scsi-mq

2017-08-04 Thread Richard W.M. Jones
https://bugzilla.redhat.com/show_bug.cgi?id=1478201 We have a libguestfs test which adds 256 virtio-scsi disks to a qemu virtual machine. The VM has 500 MB of RAM, 1 vCPU and no swap. This test has been failing for a little while. It runs out of memory during SCSI enumeration in early boot.

Re: Increased memory usage with scsi-mq

2017-08-05 Thread Richard W.M. Jones
On Sat, Aug 05, 2017 at 10:44:36AM +0200, Christoph Hellwig wrote: > On Fri, Aug 04, 2017 at 10:00:47PM +0100, Richard W.M. Jones wrote: > > I read your slides about scsi-mq and it seems like a significant > > benefit to large machines, but could the out of the box defaults b

Re: Increased memory usage with scsi-mq

2017-08-07 Thread Richard W.M. Jones
On Mon, Aug 07, 2017 at 02:11:39PM +0200, Paolo Bonzini wrote: > You could also add a module parameter to the driver, and set it to 64 on > the kernel command line (there is an example in > drivers/scsi/vmw_pvscsi.c of how to do it). [Proviso: I've not tested the performance of difference values,

Re: Increased memory usage with scsi-mq

2017-08-05 Thread Richard W.M. Jones
On Sat, Aug 05, 2017 at 03:39:54PM +0200, Christoph Hellwig wrote: > For now can you apply this testing patch to the guest kernel? > > diff --git a/drivers/scsi/virtio_scsi.c b/drivers/scsi/virtio_scsi.c > index 9be211d68b15..0cbe2c882e1c 100644 > --- a/drivers/scsi/virtio_scsi.c > +++