Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-05-03 Thread Wei Li
Hi Paolo, That will be great, I would like to hear more details about the design and implementation once you get those ready. Thanks a lot, Wei On 5/3/19, 11:05 AM, "Paolo Bonzini" wrote: On 03/05/19 10:21, Wei Li wrote: > Got it, thanks Stefan for your clarification! Hi

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-05-03 Thread Paolo Bonzini
On 03/05/19 10:21, Wei Li wrote: > Got it, thanks Stefan for your clarification! Hi Wei, Stefan and I should be posting a patch to add Linux SCSI driver batching, and an implementation for virtio-scsi. Paolo > Wei > > On 5/1/19, 9:36 AM, "Stefan Hajnoczi" wrote: > > On Mon, Apr 29,

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-05-03 Thread Wei Li
Got it, thanks Stefan for your clarification! Wei On 5/1/19, 9:36 AM, "Stefan Hajnoczi" wrote: On Mon, Apr 29, 2019 at 10:56:31AM -0700, Wei Li wrote: >Does this mean the performance could be improved via adding Batch I/O submission support in Guest driver side which will be able to

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-05-01 Thread Stefan Hajnoczi
On Mon, Apr 29, 2019 at 10:56:31AM -0700, Wei Li wrote: > Does this mean the performance could be improved via adding Batch I/O > submission support in Guest driver side which will be able to reduce the > number of virtqueue kicks? Yes, I think so. It's not obvious to me how a Linux SCSI

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-30 Thread Paolo Bonzini
On 29/04/19 15:40, Stefan Hajnoczi wrote: > This isn't fully effective since the guest driver kicks once per > request. Therefore QEMU-level batching you mentioned only works if QEMU > is slower at handling virtqueue kicks than the guest is at submitting > requests. > > I wonder if this is

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-29 Thread Wei Li
Thanks Stefan! Does this mean the performance could be improved via adding Batch I/O submission support in Guest driver side which will be able to reduce the number of virtqueue kicks? Thanks, Wei On 4/29/19, 6:40 AM, "Stefan Hajnoczi" wrote: On Fri, Apr 26, 2019 at 10:14:16AM +0200,

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-29 Thread Wei Li
Thanks Paolo for your clarification! Just wanted to double confirm, does this mean batch I/O submission won't apply to aio=threads (which is the default mode)? Thanks, Wei On 4/26/19, 9:25 PM, "Paolo Bonzini" wrote: > Thanks Stefan and Paolo for your response and advice! >

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-29 Thread Stefan Hajnoczi
On Fri, Apr 26, 2019 at 10:14:16AM +0200, Paolo Bonzini wrote: > On 23/04/19 14:04, Stefan Hajnoczi wrote: > >> In addition, does Virtio-scsi support Batch I/O Submission feature > >> which may be able to increase the IOPS via reducing the number of > >> system calls? > > > > I don't see obvious

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-26 Thread Paolo Bonzini
> Thanks Stefan and Paolo for your response and advice! > > Hi Paolo, > > As to the virtio-scsi batch I/O submission feature in QEMU which you > mentioned, is this feature turned on by default in QEMU 2.9 or there is a > tunable parameters to turn on/off the feature? Yes, it is available by

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-26 Thread Wei Li
Thanks Stefan and Paolo for your response and advice! Hi Paolo, As to the virtio-scsi batch I/O submission feature in QEMU which you mentioned, is this feature turned on by default in QEMU 2.9 or there is a tunable parameters to turn on/off the feature? Thanks, Wei On 4/26/19, 1:14 AM,

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-26 Thread Paolo Bonzini
On 23/04/19 14:04, Stefan Hajnoczi wrote: >> In addition, does Virtio-scsi support Batch I/O Submission feature >> which may be able to increase the IOPS via reducing the number of >> system calls? > > I don't see obvious batching support in drivers/scsi/virtio_scsi.c. > The Linux block layer

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-23 Thread Stefan Hajnoczi
On Mon, Apr 22, 2019 at 09:21:53PM -0700, Wei Li wrote: > 2. kvm_stat or perf record -a -e kvm:\* counters for vmexits and >interrupt injections. If these counters vary greatly between queue >sizes, then that is usually a clue. It's possible to get higher >

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-22 Thread Wei Li
Hi Stefan, I did investigation per your advices, please see inline for the details and questions. 1. Compare "iostat -dx 1" inside the guest and host. Are the I/O patterns comparable? blktrace(8) can give you even more detail on the exact I/O patterns. If the

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-17 Thread Wei Li
Sounds good, let's keep in touch. __ Thanks, Wei On 4/17/19, 5:17 AM, "Paolo Bonzini" wrote: On 17/04/19 03:38, Wei Li wrote: > Thanks Paolo for your response and clarification. > > Btw, is there any rough schedule about when are you planning to start > working on the

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-17 Thread Paolo Bonzini
On 17/04/19 03:38, Wei Li wrote: > Thanks Paolo for your response and clarification. > > Btw, is there any rough schedule about when are you planning to start > working on the multi queue feature? Once you start working on the > feature, I would like to hear more details about the design and >

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-16 Thread Wei Li
Thanks Stefan and Dongli for your feedback and advices! I will do the further investigation per your advices and get back to you later on. Thanks, -Wei On 4/16/19, 2:20 AM, "Stefan Hajnoczi" wrote: On Tue, Apr 16, 2019 at 07:23:38AM +0800, Dongli Zhang wrote: > > > On

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-16 Thread Wei Li
Thanks Paolo for your response and clarification. Btw, is there any rough schedule about when are you planning to start working on the multi queue feature? Once you start working on the feature, I would like to hear more details about the design and better understand how this feature will

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-16 Thread Paolo Bonzini
On 05/04/19 23:09, Wei Li wrote: > Thanks Stefan for your quick response! > > Hi Paolo, Could you please send us a link related to the multiqueue > feature which you are working on so that we could start getting some > details about the feature. I have never gotten to the point of multiqueue, a

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-16 Thread Stefan Hajnoczi
On Tue, Apr 16, 2019 at 07:23:38AM +0800, Dongli Zhang wrote: > > > On 4/16/19 1:34 AM, Wei Li wrote: > > Hi @Paolo Bonzini & @Stefan Hajnoczi, > > > > Would you please help confirm whether @Paolo Bonzini's multiqueue feature > > change will benefit virtio-scsi or not? Thanks! > > > > @Stefan

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-15 Thread Dongli Zhang
On 4/16/19 1:34 AM, Wei Li wrote: > Hi @Paolo Bonzini & @Stefan Hajnoczi, > > Would you please help confirm whether @Paolo Bonzini's multiqueue feature > change will benefit virtio-scsi or not? Thanks! > > @Stefan Hajnoczi, > I also spent some time on exploring the virtio-scsi multi-queue

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-15 Thread Wei Li
Hi @Paolo Bonzini & @Stefan Hajnoczi, Would you please help confirm whether @Paolo Bonzini's multiqueue feature change will benefit virtio-scsi or not? Thanks! @Stefan Hajnoczi, I also spent some time on exploring the virtio-scsi multi-queue features via num_queues parameter as below, here are

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-05 Thread Wei Li
Thanks Stefan for your quick response! Hi Paolo, Could you please send us a link related to the multiqueue feature which you are working on so that we could start getting some details about the feature. Thanks again, Wei On 4/1/19, 3:54 AM, "Stefan Hajnoczi" wrote: On Fri, Mar 29, 2019

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-04-01 Thread Stefan Hajnoczi
On Fri, Mar 29, 2019 at 08:16:36AM -0700, Wei Li wrote: > Thanks Stefan for your reply and guidance! > > We spent some time on exploring the multiple I/O Threads approach per your > feedback. Based on the perf measurement data, we did see some IOPS > improvement for multiple volumes, which is

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-03-29 Thread Wei Li
Thanks Stefan for your reply and guidance! We spent some time on exploring the multiple I/O Threads approach per your feedback. Based on the perf measurement data, we did see some IOPS improvement for multiple volumes, which is great. :) In addition, IOPS for single Volume will still be a

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-03-05 Thread Stefan Hajnoczi
On Mon, Mar 04, 2019 at 09:33:26AM -0800, Wei Li wrote: > While @Stefan mentioned about additional iothread object support of > virtio-blk, Is the feature also supported by virtio-scsi? I am trying to > exploring the perf multiple IO threads / per VM via followings: > QMP setup example to create

Re: [Qemu-devel] Following up questions related to QEMU and I/O Thread

2019-03-04 Thread Wei Li
Hi Stefan and all, I spent some time on getting familiar with QEMU and relevant concepts. My project is using QEMU 2.9 with virtio-scsi backend, and I am exploring proper way to improve the IOPS of my project. Thanks @Stefan for the response and advices! Could you please help review