qcow2-rs v0.1 and rublk-qcow2

2024-01-09 Thread Ming Lei
Hello, qcow2-rs[1] is one pure Rust library for reading/writing qcow2 image, it is based on rsd's[2] internal qcow2 implementation, but with lots of change, so far: - supports read/write on data file, backing file and compressed image - block device like interface, minimized read/write unit is

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-12 Thread Ming Lei
On Wed, Oct 12, 2022 at 10:15:28AM -0400, Stefan Hajnoczi wrote: > On Thu, 6 Oct 2022 at 06:14, Richard W.M. Jones wrote: > > > > On Tue, Oct 04, 2022 at 09:53:32AM -0400, Stefan Hajnoczi wrote: > > > qemu-nbd doesn't use io_uring to handle the backend IO, > > > > Would this be fixed by your (not

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-07 Thread Ming Lei
On Fri, Oct 07, 2022 at 07:21:51PM +0800, Yongji Xie wrote: > On Fri, Oct 7, 2022 at 6:51 PM Ming Lei wrote: > > > > On Fri, Oct 07, 2022 at 06:04:29PM +0800, Yongji Xie wrote: > > > On Thu, Oct 6, 2022 at 7:24 PM Ming Lei wrote: > > > > > > > >

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-07 Thread Ming Lei
On Thu, Oct 06, 2022 at 02:29:55PM -0400, Stefan Hajnoczi wrote: > On Thu, Oct 06, 2022 at 11:09:48PM +0800, Ming Lei wrote: > > On Thu, Oct 06, 2022 at 09:59:40AM -0400, Stefan Hajnoczi wrote: > > > On Thu, Oct 06, 2022 at 06:26:15PM +0800, Ming Lei wrote: > > > >

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-07 Thread Ming Lei
On Fri, Oct 07, 2022 at 06:04:29PM +0800, Yongji Xie wrote: > On Thu, Oct 6, 2022 at 7:24 PM Ming Lei wrote: > > > > On Wed, Oct 05, 2022 at 08:21:45AM -0400, Stefan Hajnoczi wrote: > > > On Wed, 5 Oct 2022 at 00:19, Ming Lei wrote: > > > > > > &g

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-06 Thread Ming Lei
On Thu, Oct 06, 2022 at 09:59:40AM -0400, Stefan Hajnoczi wrote: > On Thu, Oct 06, 2022 at 06:26:15PM +0800, Ming Lei wrote: > > On Wed, Oct 05, 2022 at 11:11:32AM -0400, Stefan Hajnoczi wrote: > > > On Tue, Oct 04, 2022 at 01:57:50AM +0200, Denis V. Lunev wrote: > > >

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-06 Thread Ming Lei
On Wed, Oct 05, 2022 at 08:21:45AM -0400, Stefan Hajnoczi wrote: > On Wed, 5 Oct 2022 at 00:19, Ming Lei wrote: > > > > On Tue, Oct 04, 2022 at 09:53:32AM -0400, Stefan Hajnoczi wrote: > > > On Tue, 4 Oct 2022 at 05:44, Ming Lei wrote: > > > > > > &g

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-06 Thread Ming Lei
On Wed, Oct 05, 2022 at 11:11:32AM -0400, Stefan Hajnoczi wrote: > On Tue, Oct 04, 2022 at 01:57:50AM +0200, Denis V. Lunev wrote: > > On 10/3/22 21:53, Stefan Hajnoczi wrote: > > > On Fri, Sep 30, 2022 at 05:24:11PM +0800, Ming Lei wrote: > > > > ublk-qcow2 is ava

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-04 Thread Ming Lei
On Tue, Oct 04, 2022 at 09:53:32AM -0400, Stefan Hajnoczi wrote: > On Tue, 4 Oct 2022 at 05:44, Ming Lei wrote: > > > > On Mon, Oct 03, 2022 at 03:53:41PM -0400, Stefan Hajnoczi wrote: > > > On Fri, Sep 30, 2022 at 05:24:11PM +0800, Ming Lei wrote: > > &

Re: ublk-qcow2: ublk-qcow2 is available

2022-10-04 Thread Ming Lei
On Mon, Oct 03, 2022 at 03:53:41PM -0400, Stefan Hajnoczi wrote: > On Fri, Sep 30, 2022 at 05:24:11PM +0800, Ming Lei wrote: > > ublk-qcow2 is available now. > > Cool, thanks for sharing! > > > > > So far it provides basic read/write function, and compression and

Re: [RFC v4 5/9] qemu-iotests: test new zone operations.

2022-07-27 Thread Ming Lei
+ > > +# This test only runs on Linux hosts with raw image files. > > +_supported_fmt raw > > +_supported_proto file > > +_supported_os Linux > > + > > +QEMU_IO="build/qemu-io" > > +IMG="--image-opts driver=zoned_host_device,filename=/dev/nullb0" > > +

Re: [Qemu-devel] Slow boot in QEMU with virtio-scsi disks

2018-08-11 Thread Ming Lei
e synchronize_rcu() from blk_mq_del_queue_tag_set() 5815839b3ca1 blk-mq: introduce new lock for protecting hctx->dispatch_wait 2278d69f030f blk-mq: don't pass **hctx to blk_mq_mark_tag_wait() 8ab6bb9ee8d0 blk-mq: cleanup blk_mq_get_driver_tag() Thanks, Ming Lei

Re: [Qemu-devel] [PATCH] block: fix big write

2014-12-10 Thread Ming Lei
On Wed, Dec 10, 2014 at 11:02 PM, Paolo Bonzini wrote: > > > On 10/12/2014 15:35, Ming Lei wrote: >>>> It is _not_ never happen at all, and easy to be triggered when using >>>> mkfs. >>> >>> mkfs is not something to optimize for, it's just s

Re: [Qemu-devel] [PATCH] block: fix big write

2014-12-10 Thread Ming Lei
On Wed, Dec 10, 2014 at 8:55 PM, Paolo Bonzini wrote: > > > On 10/12/2014 13:23, Ming Lei wrote: >>> > >>> > Again, we're talking of 2GB and this is something that should never >>> > happen in practice. >> Again, the 2GB limit can be av

Re: [Qemu-devel] [PATCH] block: fix big write

2014-12-10 Thread Ming Lei
On Wed, Dec 10, 2014 at 5:56 PM, Paolo Bonzini wrote: > > > On 10/12/2014 02:41, Ming Lei wrote: >> On Wed, Dec 10, 2014 at 1:45 AM, Paolo Bonzini wrote: >>> >>> >>> On 08/12/2014 08:19, Ming Lei wrote: >>>>>> >>>>>

Re: [Qemu-devel] [PATCH] block: fix big write

2014-12-09 Thread Ming Lei
On Wed, Dec 10, 2014 at 1:45 AM, Paolo Bonzini wrote: > > > On 08/12/2014 08:19, Ming Lei wrote: >>> > >>> > Alternatively, I'd accept a SCSI patch setting max_ws_blocks and friends >>> > to 2GB - 1 block. >> It should be better to not intr

[Qemu-devel] [PATCH] SCSI: splite write_zero in unit of at most INT_MAX bytes

2014-12-08 Thread Ming Lei
Currently block can not handle big write well when write size is bigger than INT_MAX, so split the write zero into smaller size of chunks to meet block's requirement. This patch fixes one WRITE SAME 16 failure in linux VM side. Cc: Max Reitz Signed-off-by: Ming Lei --- hw/scsi/scsi-d

Re: [Qemu-devel] [PATCH] block: fix big write

2014-12-07 Thread Ming Lei
On Sat, Dec 6, 2014 at 12:33 AM, Paolo Bonzini wrote: > > > On 05/12/2014 17:15, Ming Lei wrote: >> From: Ming Lei >> >> QEMU block should have supported to read/write at most >> 0x7f * 512 bytes, unfortunately INT_MAX is used to check >>

[Qemu-devel] [PATCH] block: fix big write

2014-12-05 Thread Ming Lei
From: Ming Lei QEMU block should have supported to read/write at most 0x7f * 512 bytes, unfortunately INT_MAX is used to check bytes in both bdrv_co_do_writev() and bdrv_check_byte_request(), so cause write failure if nr_sectors is equal or more than 0x40. There are still other INT_MAX

[Qemu-devel] [PATCH] scsi-disk: provide "max write same length" for block limits VPD page

2014-12-05 Thread Ming Lei
- host driver may prefer WRITE/WRITE 16, then try to parse "max write same length" Signed-off-by: Ming Lei --- hw/scsi/scsi-disk.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index 2f75d7d..b15bf4f 100644 --- a/

Re: [Qemu-devel] [RFC PATCH 3/3] linux-aio: Don't reenter request coroutine recursively

2014-12-04 Thread Ming Lei
On Thu, Dec 4, 2014 at 11:39 PM, Kevin Wolf wrote: > Am 04.12.2014 um 16:22 hat Ming Lei geschrieben: >> On Thu, Dec 4, 2014 at 10:37 PM, Kevin Wolf wrote: >> > Am 26.11.2014 um 15:46 hat Kevin Wolf geschrieben: >> >> When getting an error while submitting r

Re: [Qemu-devel] [RFC PATCH 3/3] linux-aio: Don't reenter request coroutine recursively

2014-12-04 Thread Ming Lei
On Thu, Dec 4, 2014 at 10:37 PM, Kevin Wolf wrote: > Am 26.11.2014 um 15:46 hat Kevin Wolf geschrieben: >> When getting an error while submitting requests, we must be careful to >> wake up only inactive coroutines. Therefore we must special-case the >> currently active coroutine and communicate an

Re: [Qemu-devel] [PATCH 3/7] test-coroutine: avoid overflow on 32-bit systems

2014-12-01 Thread Ming Lei
On Mon, Dec 1, 2014 at 8:41 PM, Paolo Bonzini wrote: > > > On 01/12/2014 02:28, Ming Lei wrote: >>> > - (unsigned long)(10 * duration) / maxcycles); >>> > + (unsigned long)(10.0 * duration / maxcycles)); >&g

[Qemu-devel] [PATCH v7 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-12-01 Thread Ming Lei
viewed-by: Stefan Hajnoczi Reviewed-by: Paolo Bonzini Suggested-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c | 32 1 file changed, 24 insertions(+), 8 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 53c5616..9403b17 1006

[Qemu-devel] [PATCH v7 3/3] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-12-01 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Reviewed-by: Kevin Wolf Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-)

[Qemu-devel] [PATCH v7 1/3] linux-aio: fix submit aio as a batch

2014-12-01 Thread Ming Lei
ntil aborting is handled Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c | 116 - 1 file changed, 97 insertions(+), 19 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index d92513b..53c5616 100644 ---

[Qemu-devel] [PATCH v7 0/3] linux-aio: fix batch submission

2014-12-01 Thread Ming Lei
: - code style fix and commit log fix as suggested by Benoît Canet V1: - rebase on latest QEMU master block/linux-aio.c | 139 - 1 file changed, 116 insertions(+), 23 deletions(-) Thanks, Ming Lei

Re: [Qemu-devel] [PATCH 0/7] coroutine: optimizations

2014-11-30 Thread Ming Lei
On Mon, 01 Dec 2014 08:05:17 +0100 Peter Lieven wrote: > On 01.12.2014 06:55, Ming Lei wrote: > > On Fri, Nov 28, 2014 at 10:12 PM, Paolo Bonzini wrote: > >> As discussed in the other thread, this brings speedups from > >> dropping the coroutine mutex (which se

Re: [Qemu-devel] [PATCH 0/7] coroutine: optimizations

2014-11-30 Thread Ming Lei
cost just measures this part of cost), but also some implicit/invisible part. I have some test cases which can show the problem. If someone is interested, I can post them in list. Thanks, Ming Lei

Re: [Qemu-devel] [PATCH 3/7] test-coroutine: avoid overflow on 32-bit systems

2014-11-30 Thread Ming Lei
per coroutine", > maxcycles, > duration, ops, > - (unsigned long)(10 * duration) / maxcycles); > + (unsigned long)(10.0 * duration / maxcycles)); One more single bracket. thanks, Ming Lei

Re: [Qemu-devel] [RFC PATCH 2/3] raw-posix: Convert Linux AIO submission to coroutines

2014-11-28 Thread Ming Lei
On 11/28/14, Markus Armbruster wrote: > Ming Lei writes: > >> On 11/28/14, Markus Armbruster wrote: >>> Ming Lei writes: >>> >>>> Hi Kevin, >>>> >>>> On Wed, Nov 26, 2014 at 10:46 PM, Kevin Wolf wrote: >>>>&g

Re: [Qemu-devel] [RFC PATCH 2/3] raw-posix: Convert Linux AIO submission to coroutines

2014-11-28 Thread Ming Lei
On 11/28/14, Markus Armbruster wrote: > Ming Lei writes: > >> Hi Kevin, >> >> On Wed, Nov 26, 2014 at 10:46 PM, Kevin Wolf wrote: >>> This improves the performance of requests because an ACB doesn't need to >>> be allocated on the heap any mor

Re: [Qemu-devel] [PATCH v6 1/3] linux-aio: fix submit aio as a batch

2014-11-27 Thread Ming Lei
o debug in the future. >> It's so easy to add that I'm against merging the patch without this >> protection. > > Yeah, either that or an assertion must be there. OK. Thanks, Ming Lei

Re: [Qemu-devel] [RFC PATCH 2/3] raw-posix: Convert Linux AIO submission to coroutines

2014-11-27 Thread Ming Lei
34.968s > run 5 | 30.041s | 26.263s| 35.224s | 35.000s > > TODO: Do some more serious benchmarking in VMs with less variance. > Results of a quick fio run are vaguely positive. I will do the test with Paolo's fast path approach under VM I/O situation. Thanks, Ming Lei

Re: [Qemu-devel] [PATCH v6 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-27 Thread Ming Lei
Hi Kevin, On Wed, Nov 26, 2014 at 7:27 PM, Kevin Wolf wrote: > Am 25.11.2014 um 08:23 hat Ming Lei geschrieben: >> Previously -EAGAIN is simply ignored for !s->io_q.plugged case, >> and sometimes it is easy to cause -EIO to VM, such as NVME device. >> >> This pat

Re: [Qemu-devel] [PATCH v6 1/3] linux-aio: fix submit aio as a batch

2014-11-27 Thread Ming Lei
On Wed, Nov 26, 2014 at 7:18 PM, Kevin Wolf wrote: > Am 25.11.2014 um 08:23 hat Ming Lei geschrieben: >> In the submit path, we can't complete request directly, >> otherwise "Co-routine re-entered recursively" may be caused, >> so this patch fixes the issue

Re: [Qemu-devel] [PATCH v6 1/3] linux-aio: fix submit aio as a batch

2014-11-26 Thread Ming Lei
re hasn't the potential problem in current tree, could you agree on merging it first? BTW, there isn't sort of handling for 'completion_bh' of linux aio too, :-) Thanks, Ming Lei

Re: [Qemu-devel] [PATCH v6 1/3] linux-aio: fix submit aio as a batch

2014-11-25 Thread Ming Lei
On Tue, Nov 25, 2014 at 9:08 PM, Stefan Hajnoczi wrote: > On Tue, Nov 25, 2014 at 03:23:11PM +0800, Ming Lei wrote: >> @@ -296,12 +370,14 @@ void laio_detach_aio_context(void *s_, AioContext >> *old_context) >> >> aio_set_event_notifier(old_context, &s->

[Qemu-devel] [PATCH v6 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-24 Thread Ming Lei
ted-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c | 24 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 11ac828..ac25722 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -282,8 +282,13 @@

[Qemu-devel] [PATCH v6 0/3] linux-aio: fix batch submission

2014-11-24 Thread Ming Lei
tyle fix and commit log fix as suggested by Benoît Canet V1: - rebase on latest QEMU master block/linux-aio.c | 129 +++-- 1 file changed, 106 insertions(+), 23 deletions(-) Thanks, Ming Lei

[Qemu-devel] [PATCH v6 1/3] linux-aio: fix submit aio as a batch

2014-11-24 Thread Ming Lei
ntil aborting is handled Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c | 114 - 1 file changed, 95 insertions(+), 19 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index d92513b..11ac828 100644 ---

[Qemu-devel] [PATCH v6 3/3] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-11-24 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-) diff --git a/block/linux

Re: [Qemu-devel] [PATCH v5 0/3] linux-aio: fix batch submission

2014-11-24 Thread Ming Lei
On Mon, Nov 24, 2014 at 10:31 PM, Paolo Bonzini wrote: > > > On 24/11/2014 15:29, Ming Lei wrote: >> The 1st patch fixes batch submission. >> >> The 2nd one fixes -EAGAIN for non-batch case. >> >> The 3rd one is a cleanup. >> >> This pa

[Qemu-devel] [PATCH v5 3/3] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-11-24 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-) diff --git a/block/linux

[Qemu-devel] [PATCH v5 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-24 Thread Ming Lei
off-by: Ming Lei --- block/linux-aio.c | 28 ++-- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 70312a4..0cb098d 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -282,12 +282,17 @@ static int ioq_e

[Qemu-devel] [PATCH v5 0/3] linux-aio: fix batch submission

2014-11-24 Thread Ming Lei
e changed, 107 insertions(+), 24 deletions(-) Thanks, Ming Lei

[Qemu-devel] [PATCH v5 1/3] linux-aio: fix submit aio as a batch

2014-11-24 Thread Ming Lei
ntil aborting is handled Signed-off-by: Ming Lei --- block/linux-aio.c | 116 - 1 file changed, 96 insertions(+), 20 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index d92513b..70312a4 100644 --- a/block/linux-aio.c +++

Re: [Qemu-devel] [PATCH v4 1/3] linux-aio: fix submit aio as a batch

2014-11-24 Thread Ming Lei
On Mon, Nov 24, 2014 at 7:39 PM, Paolo Bonzini wrote: > > > On 24/11/2014 12:31, Ming Lei wrote: >> +static int ioq_enqueue(struct qemu_laio_state *s, struct iocb *iocb) >> { >> unsigned int idx = s->io_q.idx; >> >> +if (unlikely(idx ==

Re: [Qemu-devel] [PATCH v4 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-24 Thread Ming Lei
On Mon, Nov 24, 2014 at 7:38 PM, Paolo Bonzini wrote: > > > On 24/11/2014 12:31, Ming Lei wrote: >> +/* don't submit until next completion for -EAGAIN of non plug case */ >> +if (unlikely(!s->io_q.plugged)) { >> +return 0; >> +} &g

[Qemu-devel] [PATCH v4 3/3] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-11-24 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Reviewed-by: Paolo Bonzini Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-) diff --git a/block/linux

[Qemu-devel] [PATCH v4 0/3] linux-aio: fix batch submission

2014-11-24 Thread Ming Lei
- rebase on latest QEMU master block/linux-aio.c | 108 + 1 file changed, 84 insertions(+), 24 deletions(-) Thanks Ming Lei

[Qemu-devel] [PATCH v4 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-24 Thread Ming Lei
off-by: Ming Lei --- block/linux-aio.c | 20 ++-- 1 file changed, 14 insertions(+), 6 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index 11fcedb..974e4f9 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -259,6 +259,11 @@ static int ioq_enqueue(

[Qemu-devel] [PATCH v4 1/3] linux-aio: fix submit aio as a batch

2014-11-24 Thread Ming Lei
n in BH context - for part of completion, update the io queue too - for case of io queue full, submit queued requests immediatelly and return failure to caller - for other failure, abort all queued requests in BH context Signed-off-by: Ming Lei --- block/lin

Re: [Qemu-devel] [PATCH v3 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-24 Thread Ming Lei
ting some I/O if the queue is getting too full. In the > * former case, instead, wait until an I/O operation is completed. > */ > if (likely(idx <= s->io_q.size * 2 / 3) || unlikely(!s->io_q.plugged) { > return 0; > } Yes, the above change need the corresponding comment, but this patch needn't since the 1st case won't reach here, :-) Thanks, Ming Lei

Re: [Qemu-devel] [PATCH v3 1/3] linux-aio: fix submit aio as a batch

2014-11-24 Thread Ming Lei
On Mon, Nov 24, 2014 at 5:47 PM, Paolo Bonzini wrote: > > > On 22/11/2014 13:16, Ming Lei wrote: >> > > +static int ioq_enqueue(struct qemu_laio_state *s, struct iocb *iocb) >> > > { >> > > unsigned int idx = s->io_q.idx; >

Re: [Qemu-devel] [PATCH 00/13] linux-aio/virtio-scsi: support AioContext wide IO submission as batch

2014-11-22 Thread Ming Lei
On Tue, Nov 18, 2014 at 9:57 PM, Paolo Bonzini wrote: > > > On 09/11/2014 08:42, Ming Lei wrote: >> This patch implements AioContext wide IO submission as batch, and >> the idea behind is very simple: >> >> - linux native aio(io_submit) supports to enqueu

Re: [Qemu-devel] [PATCH v3 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-22 Thread Ming Lei
On Tue, Nov 18, 2014 at 10:06 PM, Paolo Bonzini wrote: > > > On 06/11/2014 16:10, Ming Lei wrote: >> +/* don't submit until next completion for -EAGAIN of non plug case */ >> +if (unlikely(!s->io_q.plugged)) { >> +return 0; >> +} >

Re: [Qemu-devel] [PATCH v3 1/3] linux-aio: fix submit aio as a batch

2014-11-22 Thread Ming Lei
n ret; > } > > abort_queue(s); > ret = len; > } > > if (ret > 0) { > memmove(...) > s->io_q.idx -= ret; > } > return ret; The above is better. >> + * update io queue, for partial completion, retry will be >> + * started automatically in following completion cb. >> + */ >> +s->io_q.idx -= ret; >> + >> return ret; >> } >> >> -static void ioq_enqueue(struct qemu_laio_state *s, struct iocb *iocb) >> +static void ioq_submit_retry(void *opaque) >> +{ >> +struct qemu_laio_state *s = opaque; >> +ioq_submit(s, false); >> +} >> + >> +static int ioq_enqueue(struct qemu_laio_state *s, struct iocb *iocb) >> { >> unsigned int idx = s->io_q.idx; >> >> +if (unlikely(idx == s->io_q.size)) { >> +return -1; > > return -EAGAIN? It means the io queue is full, so the code has to fail the current request. Thanks, Ming Lei

[Qemu-devel] [PATCH] virtio-scsi: dataplane: suppress guest notification

2014-11-11 Thread Ming Lei
This patch uses vring_should_notify() to suppress guest notification, and looks notification frequency can be decreased from ~33K/sec to ~2K/sec in my test environment. Suggested-by: Stefan Hajnoczi Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-dataplane.c |7 ++- 1 file changed, 6

Re: [Qemu-devel] [PATCH v1 2/2] virtio-scsi: dataplane: notify guest as batch

2014-11-11 Thread Ming Lei
On Wed, Nov 12, 2014 at 1:28 AM, Stefan Hajnoczi wrote: > On Tue, Nov 11, 2014 at 09:17:10AM +0800, Ming Lei wrote: >> +static void notify_guest_bh(void *opaque) >> +{ >> +VirtIOSCSI *s = opaque; >> +unsigned int qid; >> +uint64_t pending = s->

Re: [Qemu-devel] [RFC PATCH 0/8] Add Generic PCI host device update

2014-11-10 Thread Ming Lei
On Tue, Nov 11, 2014 at 11:24 AM, Ming Lei wrote: > On Mon, Nov 10, 2014 at 6:00 PM, alvise rigo > wrote: >> Hi Claudio, >> >> On Fri, Nov 7, 2014 at 4:40 PM, Claudio Fontana >> wrote: >>> >>> Hi Alvise, >>> >>> I now got to te

Re: [Qemu-devel] [RFC PATCH 0/8] Add Generic PCI host device update

2014-11-10 Thread Ming Lei
tches, and it works fine on ARMv7 VM, but ARMv8 VM can't boot. 'git bisect' told me the 1st bad patch is below one: generic_pci: generate dt node after devices init Thanks, -- Ming Lei

Re: [Qemu-devel] [PATCH v1 2/2] virtio-scsi: dataplane: notify guest as batch

2014-11-10 Thread Ming Lei
On Tue, Nov 11, 2014 at 10:52 AM, Fam Zheng wrote: > On Tue, 11/11 10:29, Ming Lei wrote: >> On Tue, Nov 11, 2014 at 9:52 AM, Fam Zheng wrote: >> > On Tue, 11/11 09:17, Ming Lei wrote: >> >> It isn't necessery to notify guest each time when one request >>

Re: [Qemu-devel] [PATCH v1 2/2] virtio-scsi: dataplane: notify guest as batch

2014-11-10 Thread Ming Lei
On Tue, Nov 11, 2014 at 9:52 AM, Fam Zheng wrote: > On Tue, 11/11 09:17, Ming Lei wrote: >> It isn't necessery to notify guest each time when one request >> is completed, and it should be enough to just notify one time >> for each running of virtio_scsi_iothread_hand

[Qemu-devel] [PATCH v1 2/2] virtio-scsi: dataplane: notify guest as batch

2014-11-10 Thread Ming Lei
It isn't necessery to notify guest each time when one request is completed, and it should be enough to just notify one time for each running of virtio_scsi_iothread_handle_cmd(). This patch supresses about 30K/sec write on eventfd. Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-datapl

[Qemu-devel] [PATCH v1 0/2] virtio-scsi: dataplane: one fix and one optimization

2014-11-10 Thread Ming Lei
Hi, The 1st patch fixes an allocation problem. The 2nd one supresses writing eventfd a lot(~30K/sec in my test). V1: - use g_new() in 1/2 - avoid to dereference VIRTIO_SCSI() directly 2/2 - add build check on queue depth Thanks, Ming Lei

[Qemu-devel] [PATCH v1 1/2] virtio-scsi: dataplane: fix allocation for 'cmd_vrings'

2014-11-10 Thread Ming Lei
The size of each element should be sizeof(VirtIOSCSIVring *). Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-dataplane.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c index 855439e..df17229 100644 --- a

Re: [Qemu-devel] [PATCH 1/2] virtio-scsi-dataplane: fix allocation for 'cmd_vrings'

2014-11-10 Thread Ming Lei
On Mon, Nov 10, 2014 at 5:21 PM, Kevin Wolf wrote: > Am 10.11.2014 um 10:14 hat Ming Lei geschrieben: >> On Mon, Nov 10, 2014 at 4:24 PM, Markus Armbruster wrote: >> > Ming Lei writes: >> > >> >> The size of each element should be sizeof(VirtIOSCSIVring

Re: [Qemu-devel] [PATCH 1/2] virtio-scsi-dataplane: fix allocation for 'cmd_vrings'

2014-11-10 Thread Ming Lei
On Mon, Nov 10, 2014 at 4:24 PM, Markus Armbruster wrote: > Ming Lei writes: > >> The size of each element should be sizeof(VirtIOSCSIVring *). >> >> Signed-off-by: Ming Lei >> --- >> hw/scsi/virtio-scsi-dataplane.c |2 +- >> 1 file changed, 1 inse

[Qemu-devel] [PATCH 2/2] virtio-scsi-dataplane: notify guest as batch

2014-11-09 Thread Ming Lei
It isn't necessery to notify guest each time when one request is completed, and it should be enough to just notify one time for each running of virtio_scsi_iothread_handle_cmd(). This patch supresses about 30K/sec write on eventfd. Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-datapl

[Qemu-devel] [PATCH 1/2] virtio-scsi-dataplane: fix allocation for 'cmd_vrings'

2014-11-09 Thread Ming Lei
The size of each element should be sizeof(VirtIOSCSIVring *). Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-dataplane.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi-dataplane.c index 855439e..8a7cd9f 100644 --- a

[Qemu-devel] [PATCH 0/2] virtio-scsi-dataplane: one fix and one optimization

2014-11-09 Thread Ming Lei
The 1st patch fixes an allocation problem. The 2nd one supresses writing eventfd a lot(~30K/sec in my test). Thanks, Ming Lei

[Qemu-devel] [PATCH 13/13] virtio-scsi-dataplane: support AioContext wide IO submission as batch

2014-11-08 Thread Ming Lei
Replace previous usage with AioContext Wide approach. Signed-off-by: Ming Lei --- hw/scsi/virtio-scsi-dataplane.c |8 hw/scsi/virtio-scsi.c |2 -- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/hw/scsi/virtio-scsi-dataplane.c b/hw/scsi/virtio-scsi

[Qemu-devel] [PATCH 10/13] block/linux-aio.c: prepare for elastical resource's allocation

2014-11-08 Thread Ming Lei
Because we will support AioContext wide IO submission as batch, so IO resources should be allocated according to how many there are backends in the same AioContext. Signed-off-by: Ming Lei --- block/linux-aio.c | 38 +++--- 1 file changed, 31 insertions(+), 7

[Qemu-devel] [PATCH 11/13] block/linux-aio: reallocate I/O resources when aio attached

2014-11-08 Thread Ming Lei
y does the reallocation if there aren't any pending I/O. Signed-off-by: Ming Lei --- block/linux-aio.c | 30 ++ 1 file changed, 30 insertions(+) diff --git a/block/linux-aio.c b/block/linux-aio.c index e219b80..c5a88e8 100644 --- a/block/linux-aio.c +++ b/block/linux

[Qemu-devel] [PATCH 12/13] block: introduce bdrv_aio_io_plug() and its pair

2014-11-08 Thread Ming Lei
These two APIs are introduced for using AioContext wide IO submission as batch. Signed-off-by: Ming Lei --- block.c | 16 include/block/block.h |3 +++ 2 files changed, 19 insertions(+) diff --git a/block.c b/block.c index dacd881..0200af0 100644 --- a

[Qemu-devel] [PATCH 09/13] block/linux-aio.c: introduce laio_alloc_resource()

2014-11-08 Thread Ming Lei
This patch introduces laio_alloc_resource() for allocating resources for linux aio, then in the following patchs we can allocate IO resources just in demand. Signed-off-by: Ming Lei --- block/linux-aio.c | 55 ++--- 1 file changed, 35 insertions

[Qemu-devel] [PATCH 08/13] block/linux-aio.c: allocate events dynamically

2014-11-08 Thread Ming Lei
This patch allocates events array of 'struct qemu_laio_state' dynamically so that in the following patch we can allocate resource elasticly in case of AioContext wide IO submission as batch. Signed-off-by: Ming Lei --- block/linux-aio.c |4 +++- 1 file changed, 3 insertions(+),

[Qemu-devel] [PATCH 06/13] AioContext: introduce aio_attach_aio_bs() and its pair

2014-11-08 Thread Ming Lei
ill borrow the io queue from this 'bs', so that we can maximum IO submission as batch. aio_detach_aio_bs() will detach the 'bs' when all 'bs' in the AioContext is detached. Signed-off-by: Ming Lei --- async.c |1 + include/block/aio.h | 27

[Qemu-devel] [PATCH 05/13] block/linux-aio: pass 'BlockDriverState' to laio_attach_aio_context and its pair

2014-11-08 Thread Ming Lei
This patch introduces parameter of 'BlockDriverState' to laio_attach_aio_context() and its pair, so that it will be easier to support IO submission as batch in AioContext wide. Signed-off-by: Ming Lei --- block/linux-aio.c |6 -- block/raw-aio.h |6 -- block/raw-posi

[Qemu-devel] [PATCH 02/13] block: linux-aio: rename 'ctx' of qemu_laiocb as 'laio_state'

2014-11-08 Thread Ming Lei
So that it can be distinguished from the 'ctx' in qemu_laio_state. Signed-off-by: Ming Lei --- block/linux-aio.c |8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index b9db28e..cf8691e 100644 --- a/block/linux-aio.c +

[Qemu-devel] [PATCH 07/13] block/linux-aio: support IO submission as batch in AioContext wide

2014-11-08 Thread Ming Lei
This patch supports IO submission as batch in AioContext wide by sharing 'struct qemu_laio_state' instance among all linux-aio backend in same AioContext. Signed-off-by: Ming Lei --- block/linux-aio.c | 36 ++-- 1 file changed, 34 insertions(+), 2

[Qemu-devel] [PATCH 04/13] block/linux-aio: do more things in laio_state_alloc() and its pair

2014-11-08 Thread Ming Lei
Now lifetime of 'completion_bh', io queue and io context is same, so move their allocation into laio_state_alloc() and their releasing into laio_state_free(). Signed-off-by: Ming Lei --- block/linux-aio.c | 26 -- 1 file changed, 12 insertions(+), 14 deletion

[Qemu-devel] [PATCH 03/13] block/linux-aio: allocate 'struct qemu_laio_state' dynamically

2014-11-08 Thread Ming Lei
This patch allocates 'struct qemu_laio_state' in aio attach, and frees it in aio detach, so that in the following patch we can share one same instance of the structure among multiple linux-aio backend in same AioContext. Signed-off-by: Ming Lei --- block/linux-ai

[Qemu-devel] [PATCH 01/13] block/linux-aio: allocate io queue dynamically

2014-11-08 Thread Ming Lei
This patch allocates io queue dynamically so that we can support aio_context wide io queue in the following patch. Signed-off-by: Ming Lei --- block/linux-aio.c | 66 + 1 file changed, 41 insertions(+), 25 deletions(-) diff --git a/block

[Qemu-devel] [PATCH 00/13] linux-aio/virtio-scsi: support AioContext wide IO submission as batch

2014-11-08 Thread Ming Lei
| 27 + include/block/block.h |3 + 9 files changed, 259 insertions(+), 59 deletions(-) Thanks, Ming Lei

[Qemu-devel] [PATCH v3 1/3] linux-aio: fix submit aio as a batch

2014-11-06 Thread Ming Lei
- for part of completion, also update the io queue - for other failure, return the failure if in enqueue path, otherwise, abort all queued I/O Signed-off-by: Ming Lei --- block/linux-aio.c | 101 + 1 file changed, 79 in

[Qemu-devel] [PATCH v3 2/3] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-11-06 Thread Ming Lei
off-by: Ming Lei --- block/linux-aio.c | 22 +- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index f66e8ad..f5ca41d 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -263,6 +263,11 @@ static int ioq_enqueue(

[Qemu-devel] [PATCH v3 0/3] linux-aio: fix batch submission

2014-11-06 Thread Ming Lei
The 1st patch fixes batch submission. The 2nd one fixes -EAGAIN for non-batch case. The 3rd one is a cleanup. This patchset is splitted from previous patchset(dataplane: optimization and multi virtqueue support), as suggested by Stefan. v3: - rebase on QEMU master v2: - code sty

[Qemu-devel] [PATCH v3 3/3] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-11-06 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c in

Re: [Qemu-devel] [PATCH v2 0/4] linux-aio: fix batch submission

2014-10-15 Thread Ming Lei
Hi Stefan and Guys, On Thu, Sep 4, 2014 at 6:27 PM, Ming Lei wrote: > The 1st patch fixes batch submission. > > The 2nd one fixes -EAGAIN for non-batch case. > > The 3rd one is a cleanup. > > The 4th one increase max event to 256 for supporting the comming > multi virt-

Re: [Qemu-devel] [PATCH v2 0/4] linux-aio: fix batch submission

2014-09-09 Thread Ming Lei
Hi Paolo, Stefan and Kevin, On Fri, Sep 5, 2014 at 12:27 AM, Ming Lei wrote: > The 1st patch fixes batch submission. > > The 2nd one fixes -EAGAIN for non-batch case. > > The 3rd one is a cleanup. > > The 4th one increase max event to 256 for supporting the comming > mu

[Qemu-devel] [PATCH v2 4/4] linux-aio: increase max event to 256

2014-09-04 Thread Ming Lei
This patch increases max event to 256 for the coming virtio-blk multi virtqueue support. Signed-off-by: Ming Lei --- block/linux-aio.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index f45a142..5d565ad 100644 --- a/block/linux

[Qemu-devel] [PATCH v2 1/4] linux-aio: fix submit aio as a batch

2014-09-04 Thread Ming Lei
g completion cb - for part of completion, also update the io queue - for other failure, return the failure if in enqueue path, otherwise, abort all queued I/O Signed-off-by: Ming Lei --- block/linux-aio.c | 106 - 1 file changed

[Qemu-devel] [PATCH v2 3/4] linux-aio: remove 'node' from 'struct qemu_laiocb'

2014-09-04 Thread Ming Lei
No one uses the 'node' field any more, so remove it from 'struct qemu_laiocb', and this can save 16byte for the struct on 64bit arch. Signed-off-by: Ming Lei --- block/linux-aio.c |1 - 1 file changed, 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c in

[Qemu-devel] [PATCH v2 2/4] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-09-04 Thread Ming Lei
off-by: Ming Lei --- block/linux-aio.c | 22 +- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index a06576d..20a87ec 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -276,6 +276,11 @@ static int ioq_enqueue(

[Qemu-devel] [PATCH v2 0/4] linux-aio: fix batch submission

2014-09-04 Thread Ming Lei
-aio.c | 131 + 1 file changed, 103 insertions(+), 28 deletions(-) Thanks, -- Ming Lei

Re: [Qemu-devel] [PATCH 1/4] linux-aio: fix submit aio as a batch

2014-09-04 Thread Ming Lei
On Thu, Sep 4, 2014 at 10:59 PM, Benoît Canet wrote: > The Thursday 14 Aug 2014 à 17:41:41 (+0800), Ming Lei wrote : >> In the enqueue path, we can't complete request, otherwise >> "Co-routine re-entered recursively" may be caused, so this >> patch fixes th

[Qemu-devel] [PATCH v1 2/4] linux-aio: handling -EAGAIN for !s->io_q.plugged case

2014-09-03 Thread Ming Lei
off-by: Ming Lei --- block/linux-aio.c | 22 +- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index ff66d1e..a979331 100644 --- a/block/linux-aio.c +++ b/block/linux-aio.c @@ -275,6 +275,11 @@ static int ioq_enqueue(

[Qemu-devel] [PATCH v1 4/4] linux-aio: increase max event to 256

2014-09-03 Thread Ming Lei
This patch increases max event to 256 for the comming virtio-blk multi virtqueue support. Signed-off-by: Ming Lei --- block/linux-aio.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/linux-aio.c b/block/linux-aio.c index ee125bc..4536106 100644 --- a/block/linux

  1   2   3   4   >