[Qemu-devel] [PATCH 2/2] block: Add copy offloading trace points

2018-07-04 Thread Fam Zheng
A few trace points that can help reveal what is happening in a copy offloading I/O path. Signed-off-by: Fam Zheng --- block/file-posix.c | 2 ++ block/io.c | 2 ++ block/iscsi.c | 3 +++ block/trace-events | 6 ++ 4 files changed, 13 insertions(+) diff --git a/block/file

[Qemu-devel] [PATCH 1/2] block: Fix dst total_sectors after copy offloading

2018-07-04 Thread Fam Zheng
This was noticed by the new image fleecing tests case 222. The issue is apparent and we should just do the same right things as in bdrv_aligned_pwritev. Reported-by: John Snow Signed-off-by: Fam Zheng --- block/io.c | 4 1 file changed, 4 insertions(+) diff --git a/block/io.c b/block

[Qemu-devel] [PATCH 0/2] block: Fix dst reading after tail copy offloading

2018-07-04 Thread Fam Zheng
ic in the normal write code and update bs->total_sectors after I/O is done. While at it, add a few convenient trace points to aid future debug experiences in the topic. Fam Zheng (2): block: Fix dst total_sectors after copy offloading block: Add copy offloading trace points block/file-posix.c

Re: [Qemu-devel] [PATCH v2] module: Use QEMU_MODULE_PATH as a search path

2018-07-03 Thread Fam Zheng
On Tue, 07/03 09:38, ryang wrote: > The current paths for modules are CONFIG_QEMU_MODDIR and paths relative > to the executable. Qemu and its modules can be installed and executed in > paths that are different from these search paths. This change allows > a search path to be specified by

Re: [Qemu-devel] patchew header docs [was: Re: [PATCH 0/6] hyperv: refactor HvSintRoute management]

2018-07-03 Thread Fam Zheng
On Tue, 07/03 07:48, Eric Blake wrote: > On 07/03/2018 03:00 AM, Fam Zheng wrote: > > On Tue, 07/03 10:47, Roman Kagan wrote: > > > On Mon, Jul 02, 2018 at 03:55:33PM -0500, Eric Blake wrote: > > > > On 07/02/2018 11:58 AM, Roman Kagan wrote: > > >

Re: [Qemu-devel] patchew header docs [was: Re: [PATCH 0/6] hyperv: refactor HvSintRoute management]

2018-07-03 Thread Fam Zheng
On Tue, 07/03 10:47, Roman Kagan wrote: > On Mon, Jul 02, 2018 at 03:55:33PM -0500, Eric Blake wrote: > > On 07/02/2018 11:58 AM, Roman Kagan wrote: > > > This series modifies the management of HvSintRoute, which is an > > > important building block in Hyper-V emulation infrastructure, to make it

[Qemu-devel] [PATCH v4 3/3] backup: Use copy offloading

2018-07-02 Thread Fam Zheng
request fails because of, for example, the offset is beyond EOF, but another may well be accepted by the protocol layer. This will be implemented separately. Reviewed-by: Stefan Hajnoczi Signed-off-by: Fam Zheng --- block/backup.c | 150 - block/trace

[Qemu-devel] [PATCH v4 1/3] block: Fix parameter checking in bdrv_co_copy_range_internal

2018-07-02 Thread Fam Zheng
off-by: Fam Zheng --- block/io.c | 29 +++-- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/block/io.c b/block/io.c index 7035b78a20..b8845708d7 100644 --- a/block/io.c +++ b/block/io.c @@ -2897,18 +2897,11 @@ static int coroutine

[Qemu-devel] [PATCH v4 2/3] block: Honour BDRV_REQ_NO_SERIALISING in copy range

2018-07-02 Thread Fam Zheng
This semantics is needed by drive-backup so implement it before using this API there. Reviewed-by: Stefan Hajnoczi Signed-off-by: Fam Zheng --- block/io.c| 6 -- include/block/block.h | 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block

[Qemu-devel] [PATCH v4 0/3] backup: Use copy offloading

2018-07-02 Thread Fam Zheng
to use the bounce buffer as well as speeding up the copy operation when the backend supports it. v3: Don't forget coroutine_fn. [Stefan] Don't reset job->use_copy_range redundantly. [Stefan] v2: Use helper functions. [Stefan] Fam Zheng (3): block: Fix parameter check

Re: [Qemu-devel] [PATCH 1/4] tests/vm: Support proxy / corporate firewall

2018-07-02 Thread Fam Zheng
On Mon, 07/02 12:11, Philippe Mathieu-Daudé wrote: > Hi Fam, > > On 07/02/2018 04:12 AM, Fam Zheng wrote: > > On Thu, 06/28 12:35, Philippe Mathieu-Daudé wrote: > >> If ftp_proxy/http_proxy/https_proxy standard environment variables > >> are available, pass them t

Re: [Qemu-devel] [PATCH 0/4] block: Trivial fixes in offloading code

2018-07-02 Thread Fam Zheng
On Mon, 07/02 14:35, Max Reitz wrote: > On 2018-07-02 04:58, Fam Zheng wrote: > > These are the low priority ones spotted by Kevin and Max last week. > > > > Fam Zheng (4): > > qcow2: Drop unused cluster_data > > file-posix: Fix fd_open check in raw_c

Re: [Qemu-devel] [PATCH 1/2] vmdk: Fix possible segfault with non-VMDK backing

2018-07-02 Thread Fam Zheng
->format_name, "vmdk")) { > +/* Backing file is not in vmdk format, so it does not have > + * a CID, which makes the overlay's parent CID invalid */ > +return 0; > +} > + Reviewed-by: Fam Zheng > if (vmdk_read_cid(p_bs, 0, _pcid) != 0) { > /* read failure: report as not valid */ > return 0; > -- > 2.17.1 >

Re: [Qemu-devel] [PATCH v3 0/2] backup: Use copy offloading

2018-07-02 Thread Fam Zheng
Jeff: ping? Can we have this in 3.0? On Tue, 06/05 22:06, Fam Zheng wrote: > Based-on: <20180529055959.32002-1-f...@redhat.com> > ([PATCH v7 00/10] qemu-img convert with copy offloading) > > This enhances the backup job to make use of the copy offloading API. It > eliminates

Re: [Qemu-devel] [PATCH 0/4] tests/vm: various trivial fixes

2018-07-02 Thread Fam Zheng
On Thu, 06/28 12:35, Philippe Mathieu-Daudé wrote: > Hi, > > These are various fixes I added to have the VM tests working for me. Queued the last two patches. Thanks. Fam

Re: [Qemu-devel] [PATCH 2/4] tests/vm: Add a dependency to qemu-img

2018-07-02 Thread Fam Zheng
On Thu, 06/28 12:35, Philippe Mathieu-Daudé wrote: > Before the first use, the VM image are resized with qemu-img. > > Add a dependency to the qemu-img tool to fix: > > $ make vm-build-ubuntu.i386 > Traceback (most recent call last): > File "source/qemu/tests/vm/basevm.py", line 236, in

Re: [Qemu-devel] [PATCH 1/4] tests/vm: Support proxy / corporate firewall

2018-07-02 Thread Fam Zheng
On Thu, 06/28 12:35, Philippe Mathieu-Daudé wrote: > If ftp_proxy/http_proxy/https_proxy standard environment variables > are available, pass them to the vm images. > > As per 06cc3551714: > This is required when building behind corporate proxy/firewall, but > also help when using local cache

Re: [Qemu-devel] [PATCH v2 3/3] qemu-iotests: Image fleecing test case 222

2018-07-02 Thread Fam Zheng
On Fri, 06/29 17:04, John Snow wrote: > Mine was indeed inspired by Fam's version, but I opted to rewrite it > instead of continue with the python unit tests approach. This is why I > lost the commit message, existing copyright text, etc. > > I can re-add him as author credit to my version, or at

Re: [Qemu-devel] [PATCH v2 2/3] block/fleecing-filter: new filter driver for fleecing

2018-07-02 Thread Fam Zheng
On Fri, 06/29 12:24, Eric Blake wrote: > On 06/29/2018 10:15 AM, Vladimir Sementsov-Ogievskiy wrote: > > We need to synchronize backup job with reading from fleecing image > > like it was done in block/replication.c. > > > > Otherwise, the following situation is theoretically possible: > > > >

Re: [Qemu-devel] [PULL 0/2] Block patches

2018-07-02 Thread Fam Zheng
On Sun, 07/01 07:58, no-re...@patchew.org wrote: > /var/tmp/patchew-tester-tmp-x_t1hkpk/src/block/file-posix.c:1853:13: error: > redefinition of ‘raw_aio_attach_aio_context’ > static void raw_aio_attach_aio_context(BlockDriverState *bs, > ^~ >

[Qemu-devel] [PATCH 3/4] qcow2: Drop unreachable break

2018-07-01 Thread Fam Zheng
Reported-by: Max Reitz Signed-off-by: Fam Zheng --- block/qcow2.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/qcow2.c b/block/qcow2.c index 3177c72de0..1a2e498a3c 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3292,7 +3292,6 @@ qcow2_co_copy_range_from(BlockDriverState *bs

[Qemu-devel] [PATCH 2/4] file-posix: Fix fd_open check in raw_co_copy_range_to

2018-07-01 Thread Fam Zheng
One of them is a typo. But update both to be more readable. Reported-by: Kevin Wolf Signed-off-by: Fam Zheng --- block/file-posix.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/file-posix.c b/block/file-posix.c index 829ee538d8..e04e464e72 100644 --- a/block/file

[Qemu-devel] [PATCH 4/4] raw: Drop superfluous semicolon

2018-07-01 Thread Fam Zheng
Reported-by: Max Reitz Signed-off-by: Fam Zheng --- block/raw-format.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/raw-format.c b/block/raw-format.c index b78da564d4..8e648a5666 100644 --- a/block/raw-format.c +++ b/block/raw-format.c @@ -177,7 +177,7 @@ static

[Qemu-devel] [PATCH 0/4] block: Trivial fixes in offloading code

2018-07-01 Thread Fam Zheng
These are the low priority ones spotted by Kevin and Max last week. Fam Zheng (4): qcow2: Drop unused cluster_data file-posix: Fix fd_open check in raw_co_copy_range_to qcow2: Drop unreachable break raw: Drop superfluous semicolon block/file-posix.c | 2 +- block/qcow2.c | 3

[Qemu-devel] [PATCH 1/4] qcow2: Drop unused cluster_data

2018-07-01 Thread Fam Zheng
Reported-by: Max Reitz Signed-off-by: Fam Zheng --- block/qcow2.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 2f9e58e0c4..3177c72de0 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3338,7 +3338,6 @@ qcow2_co_copy_range_to(BlockDriverState *bs

Re: [Qemu-devel] [RFC PATCH 5/8] docker: Restrict the 'travis' job to the Travis image

2018-06-29 Thread Fam Zheng
On Thu, 06/28 13:46, Philippe Mathieu-Daudé wrote: > We can still run any test in Travis. > > Signed-off-by: Philippe Mathieu-Daudé > --- > tests/docker/Makefile.include | 10 -- > 1 file changed, 8 insertions(+), 2 deletions(-) > > diff --git a/tests/docker/Makefile.include

[Qemu-devel] [PATCH 3/3] file-posix: Fix EINTR handling

2018-06-29 Thread Fam Zheng
, so that it could still retry with other byte ranges, whereas it shouldn't retry anymore upon ENOSYS. Signed-off-by: Fam Zheng --- block/file-posix.c | 17 + 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c index 07bb061fe4

[Qemu-devel] [PATCH 2/3] iscsi: Don't blindly use designator length in response for memcpy

2018-06-29 Thread Fam Zheng
Per SCSI definition the designator_length we receive from INQUIRY is 8, 12 or at most 16, but we should be careful because the remote iscsi target may misbehave, otherwise we could have a buffer overflow. Reported-by: Max Reitz Signed-off-by: Fam Zheng --- block/iscsi.c | 2 +- 1 file changed

[Qemu-devel] [PATCH 1/3] qcow2: Fix src_offset in copy offloading

2018-06-29 Thread Fam Zheng
Not updating src_offset will result in wrong data being written to dst image. Reported-by: Max Reitz Signed-off-by: Fam Zheng --- block/qcow2.c | 1 + tests/qemu-iotests/063 | 9 + tests/qemu-iotests/063.out | 12 3 files changed, 22 insertions(+) diff

[Qemu-devel] [PATCH 0/3] block: A few more copy offloading fixes

2018-06-29 Thread Fam Zheng
These are unfortunately more serious than the previous two fixes but the patches are not complicated. Fam Zheng (3): qcow2: Fix src_offset in copy offloading iscsi: Don't blindly use designator length in response for memcpy file-posix: Fix EINTR handling block/file-posix.c | 17

[Qemu-devel] [PATCH 1/2] qcow2: Remove dead check on !ret

2018-06-26 Thread Fam Zheng
In the beginning of the function, we initialize the local variable to 0, and in the body of the function, we check the assigned values and exit the loop immediately. So here it can never be non-zero. Reported-by: Kevin Wolf Signed-off-by: Fam Zheng --- block/qcow2.c | 2 +- 1 file changed, 1

[Qemu-devel] [PATCH 0/2] block: Two copy offloading corrections

2018-06-26 Thread Fam Zheng
Fam Zheng (2): qcow2: Remove dead check on !ret block: Move request tracking to children in copy offloading block/io.c| 59 --- block/qcow2.c | 2 +- 2 files changed, 29 insertions(+), 32 deletions(-) -- 2.17.1

[Qemu-devel] [PATCH 2/2] block: Move request tracking to children in copy offloading

2018-06-26 Thread Fam Zheng
it. Reported-by: Kevin Wolf Signed-off-by: Fam Zheng --- block/io.c | 59 ++ 1 file changed, 28 insertions(+), 31 deletions(-) diff --git a/block/io.c b/block/io.c index ef4fedd364..585008a6fb 100644 --- a/block/io.c +++ b/block/io.c

Re: [Qemu-devel] [PATCH v3 2/2] block/file-posix: reconfigure aio on iothread start

2018-06-21 Thread Fam Zheng
On Thu, 06/21 15:21, Nishanth Aravamudan wrote: > When the AioContext changes, we need to associate a LinuxAioState with > the new AioContext. Use the bdrv_attach_aio_context callback and call > the new aio_setup_linux_aio(), which will allocate a new AioContext if > needed, and return errors on

Re: [Qemu-devel] [PATCH v3 1/2] linux-aio: properly bubble up errors from initialization

2018-06-21 Thread Fam Zheng
On Thu, 06/21 15:21, Nishanth Aravamudan wrote: > laio_init() can fail for a couple of reasons, which will lead to a NULL > pointer dereference in laio_attach_aio_context(). > > To solve this, add a aio_setup_linux_aio() function which is called > early in raw_open_common. If this fails,

Re: [Qemu-devel] [PATCH] nvme: Support image creation

2018-06-13 Thread Fam Zheng
On Wed, 06/13 10:06, Kevin Wolf wrote: > Am 13.06.2018 um 09:46 hat Fam Zheng geschrieben: > > Similar to the host_device's implementation, we check the requested > > length against the namespace size. > > > > Truncation is necessary to make qcow2 creation work. > &

[Qemu-devel] [PATCH] nvme: Support image creation

2018-06-13 Thread Fam Zheng
Similar to the host_device's implementation, we check the requested length against the namespace size. Truncation is necessary to make qcow2 creation work. Signed-off-by: Fam Zheng --- block/nvme.c | 72 1 file changed, 72 insertions

[Qemu-devel] [PATCH] nvme: Reset s->nr_queues upon open failure

2018-06-13 Thread Fam Zheng
It is wrong to leave this field as 1, as nvme_close() called in the error handling code in nvme_file_open() will use it and try to free s->queues again. Clear the fields to avoid double-free. Cc: qemu-sta...@nongnu.org Signed-off-by: Fam Zheng --- block/nvme.c | 2 ++ 1 file changed

Re: [Qemu-devel] [PATCH] travis: display config.log when configure fails

2018-06-12 Thread Fam Zheng
igure ${CONFIG} --extra-cflags="-g3 -O0 -fsanitize=thread > -fuse-ld=gold" || (cat config.log && exit 1) > -- > 2.17.0 > Makes sense. this fixes the status code of before_script, though previously as we mask the error with the "|| cat config.log", make will still fail. Reviewed-by: Fam Zheng

Re: [Qemu-devel] Is there a way to package QEMU binaries?

2018-06-12 Thread Fam Zheng
On Tue, 06/12 14:52, Peter Xu wrote: > On Tue, Jun 12, 2018 at 02:41:19PM +0800, Fam Zheng wrote: > > On Tue, 06/12 14:24, Peter Xu wrote: > > > Hi, > > > > > > For example, I wanted to compile QEMU once and install it on multiple > > > syste

Re: [Qemu-devel] Is there a way to package QEMU binaries?

2018-06-12 Thread Fam Zheng
On Tue, 06/12 14:24, Peter Xu wrote: > Hi, > > For example, I wanted to compile QEMU once and install it on multiple > systems. What would be the suggested way to do so? > > Is there something similar to "make bin-rpmpkg" for Linux? > > Thanks in advance, No. The big question is the

Re: [Qemu-devel] question: a dead loop in qemu when do blockJobAbort and vm suspend coinstantaneously

2018-06-11 Thread Fam Zheng
pointer reference in > > bdrv_detach_aio_context. The code is below: > > > > void bdrv_detach_aio_context(BlockDriverState *bs) > > { > >     . > > > > QLIST_FOREACH_SAFE(baf, >aio_notifiers, list, baf_tmp) { > >     if (baf->deleted) { > >        

Re: [Qemu-devel] [PATCH] util/async: avoid NULL pointer dereference

2018-06-11 Thread Fam Zheng
On Mon, 06/11 15:04, Jie Wang wrote: > if laio_init create linux_aio failed and return NULL, NULL pointer > dereference will occur when laio_attach_aio_context dereference > linux_aio in aio_get_linux_aio, so add assert to avoid it. > > Signed-off-by: Jie Wang > --- > util/async.c | 1 + > 1

Re: [Qemu-devel] question: a dead loop in qemu when do blockJobAbort and vm suspend coinstantaneously

2018-06-10 Thread Fam Zheng
On Sat, 06/09 17:10, l00284672 wrote: > Hi, I found a dead loop in qemu when do blockJobAbort and vm suspend > coinstantaneously. > > The qemu bt is below: > > #0  0x7ff58b53af1f in ppoll () from /lib64/libc.so.6 > #1  0x007fdbd9 in ppoll (__ss=0x0, __timeout=0x7ffcf7055390, >

[Qemu-devel] [PATCH 4/6] block-backend: Add blk_aio_copy_range

2018-06-08 Thread Fam Zheng
Signed-off-by: Fam Zheng --- block/block-backend.c | 36 ++ include/sysemu/block-backend.h | 4 2 files changed, 40 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index e20a204bee..36d928e13d 100644 --- a/block/block

[Qemu-devel] [PATCH 6/6] mirror: Use copy offloading

2018-06-08 Thread Fam Zheng
This makes the mirror job to try offloaded copy. If it fails, error action will not be taken yet, instead the failed cluster and all the subsequent ones will fall back to bounce buffer. Signed-off-by: Fam Zheng --- block/mirror.c | 71 +- block

[Qemu-devel] [PATCH 3/6] block-backend: Refactor AIO emulation

2018-06-08 Thread Fam Zheng
differentiation in parameters, refactor a bit. Move the per-request fields to a union and create one struct for each type. While at it also move the bytes parameter from BlkAioEmAIOCB to BlkRwCo. Signed-off-by: Fam Zheng --- block/block-backend.c | 211 +++--- 1 file

[Qemu-devel] [PATCH 2/6] block: Check if block drivers can do copy offloading

2018-06-08 Thread Fam Zheng
This avoids the wasteful cluster allocation in qcow2 before actually trying an unsupported copy range call, for example. Signed-off-by: Fam Zheng --- block.c | 12 block/file-posix.c| 9 + block/io.c| 3 +++ block/iscsi.c

[Qemu-devel] [PATCH 5/6] block: Add backing passthrough implementations for copy_range

2018-06-08 Thread Fam Zheng
Similar to bdrv_co_block_status_from_backing we add the two passthrough callbacks for copy_range. This will be used by the block driver filters so that they can support copy offloading. Signed-off-by: Fam Zheng --- block/io.c| 24 include/block

[Qemu-devel] [PATCH 1/6] file-posix: Fix EINTR handling

2018-06-08 Thread Fam Zheng
EINTR should be checked against errno, not ret. While fixing the bug, collecting the branches with a switch block. Signed-off-by: Fam Zheng --- block/file-posix.c | 20 ++-- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/block/file-posix.c b/block/file-posix.c

[Qemu-devel] [PATCH 0/6] mirror: Use copy offloading

2018-06-08 Thread Fam Zheng
This is the third part of copy offloading work. The first patches are fixes and improvements in preparation for enabling mirror job. The last patch does a similar change to the backup patch: it inserts a blk_aio_copy_range call before the usual bounce buffer code in mirror_iteration. Fam Zheng (6

Re: [Qemu-devel] [PULL 0/6] VFIO updates 2018-06-05

2018-06-07 Thread Fam Zheng
On Wed, 06/06 03:29, no-re...@patchew.org wrote: > Hi, > > This series failed docker-mingw@fedora build test. Please find the testing > commands and > their output below. If you have Docker installed, you can probably reproduce > it > locally. > > Type: series > Message-id:

[Qemu-devel] 100% host cpu with gtk3

2018-06-06 Thread Fam Zheng
Hi Gerd, When using the gtk frontend, it seems there is a busy loop in libgtk repeatedly calling recvmsg, causing 100% cpu on the main thread. This started to appear after I upgraded to Fedora 28. Is this a known problem? Any hints on how to track it down? Fam

[Qemu-devel] [PATCH v3 2/2] backup: Use copy offloading

2018-06-05 Thread Fam Zheng
The implementation is similar to the 'qemu-img convert'. In the beginning of the job, offloaded copy is attempted. If it fails, further I/O will go through the existing bounce buffer code path. Signed-off-by: Fam Zheng --- block/backup.c | 150

[Qemu-devel] [PATCH v3 0/2] backup: Use copy offloading

2018-06-05 Thread Fam Zheng
pports it. v3: Don't forget coroutine_fn. [Stefan] Don't reset job->use_copy_range redundantly. [Stefan] v2: Use helper functions. [Stefan] Fam Zheng (2): block: Honour BDRV_REQ_NO_SERIALISING in copy range backup: Use copy offloading block/backup.c

[Qemu-devel] [PATCH v3 1/2] block: Honour BDRV_REQ_NO_SERIALISING in copy range

2018-06-05 Thread Fam Zheng
This semantics is needed by drive-backup so implement it before using this API there. Signed-off-by: Fam Zheng --- block/io.c| 6 -- include/block/block.h | 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index b7beaeeb9f

Re: [Qemu-devel] [PATCH v2 2/2] backup: Use copy offloading

2018-06-05 Thread Fam Zheng
On Tue, 06/05 13:22, Stefan Hajnoczi wrote: > On Tue, Jun 05, 2018 at 04:07:10PM +0800, Fam Zheng wrote: > > The implementation is similar to the 'qemu-img convert'. In the > > beginning of the job, offloaded copy is attempted. If it fails, further > > I/O will go throug

[Qemu-devel] [PATCH v2 0/2] backup: Use copy offloading

2018-06-05 Thread Fam Zheng
pports it. v2: Use helper functions. [Stefan] Fam Zheng (2): block: Honour BDRV_REQ_NO_SERIALISING in copy range backup: Use copy offloading block/backup.c| 151 ++ block/io.c| 6 +- block/trace-events| 1 + include/block/b

[Qemu-devel] [PATCH v2 2/2] backup: Use copy offloading

2018-06-05 Thread Fam Zheng
The implementation is similar to the 'qemu-img convert'. In the beginning of the job, offloaded copy is attempted. If it fails, further I/O will go through the existing bounce buffer code path. Signed-off-by: Fam Zheng --- block/backup.c | 151

[Qemu-devel] [PATCH v2 1/2] block: Honour BDRV_REQ_NO_SERIALISING in copy range

2018-06-05 Thread Fam Zheng
This semantics is needed by drive-backup so implement it before using this API there. Signed-off-by: Fam Zheng --- block/io.c| 6 -- include/block/block.h | 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index b7beaeeb9f

Re: [Qemu-devel] [PATCH v1 5/7] docker: Update fedora image to 28

2018-06-04 Thread Fam Zheng
On Mon, 06/04 15:51, Alex Bennée wrote: > From: Fam Zheng > > Signed-off-by: Fam Zheng > Signed-off-by: Alex Bennée > --- > tests/docker/dockerfiles/fedora.docker | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/docker/dockerfiles/fedo

[Qemu-devel] [PULL 1/2] tests/docker/Makefile.include: handle empty TARGET_LIST

2018-06-04 Thread Fam Zheng
ben...@linaro.org> Signed-off-by: Fam Zheng --- rules.mak | 3 +++ tests/docker/Makefile.include | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/rules.mak b/rules.mak index 04c7f74d07..bbb2667928 100644 --- a/rules.mak +++ b/rules.mak @@ -1,

[Qemu-devel] [PULL 0/2] Docker patches

2018-06-04 Thread Fam Zheng
. Alex Bennée (1): tests/docker/Makefile.include: handle empty TARGET_LIST Fam Zheng (1): docker: Update fedora image to 28 rules.mak | 3 +++ tests/docker/Makefile.include | 2 +- tests

[Qemu-devel] [PULL 2/2] docker: Update fedora image to 28

2018-06-04 Thread Fam Zheng
Signed-off-by: Fam Zheng Message-Id: <20180601023557.9770-1-f...@redhat.com> --- tests/docker/dockerfiles/fedora.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index b706

Re: [Qemu-devel] [PATCH] file-posix: Consolidate the locking error message

2018-06-01 Thread Fam Zheng
On Fri, 06/01 15:00, Daniel P. Berrangé wrote: > On Fri, Jun 01, 2018 at 09:33:59PM +0800, Fam Zheng wrote: > > On Fri, 06/01 13:43, Daniel P. Berrangé wrote: > > > On Fri, Jun 01, 2018 at 05:18:35PM +0800, Fam Zheng wrote: > > > > When hot-plugging a block device fai

Re: [Qemu-devel] [PATCH] file-posix: Consolidate the locking error message

2018-06-01 Thread Fam Zheng
On Fri, 06/01 07:32, Eric Blake wrote: > On 06/01/2018 04:18 AM, Fam Zheng wrote: > > When hot-plugging a block device fails due to image locking errors, > > users won't see the helpful 'Is another process using the image?' > > message in QMP because currently the error hin

Re: [Qemu-devel] [PATCH] file-posix: Consolidate the locking error message

2018-06-01 Thread Fam Zheng
On Fri, 06/01 13:43, Daniel P. Berrangé wrote: > On Fri, Jun 01, 2018 at 05:18:35PM +0800, Fam Zheng wrote: > > When hot-plugging a block device fails due to image locking errors, > > users won't see the helpful 'Is another process using the image?' > > message in QMP becaus

Re: [Qemu-devel] [PATCH v8 00/11] qemu-img convert with copy offloading

2018-06-01 Thread Fam Zheng
On Fri, 06/01 10:37, Stefan Hajnoczi wrote: > On Thu, May 31, 2018 at 11:45:17PM -0700, no-re...@patchew.org wrote: > > /var/tmp/patchew-tester-tmp-2l7s8dte/src/block/iscsi.c: In function > > ‘iscsi_populate_target_desc’: > > /var/tmp/patchew-tester-tmp-2l7s8dte/src/block/iscsi.c:2242:15: error:

[Qemu-devel] [PATCH v9 10/10] qemu-img: Convert with copy offloading

2018-06-01 Thread Fam Zheng
The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Signed-off-by: Fam

[Qemu-devel] [PATCH v9 08/10] iscsi: Implement copy offloading

2018-06-01 Thread Fam Zheng
Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219 +++ include/scsi

[Qemu-devel] [PATCH v9 04/10] qcow2: Implement copy offloading

2018-06-01 Thread Fam Zheng
igned-off-by: Fam Zheng --- block/qcow2.c | 229 +++--- 1 file changed, 199 insertions(+), 30 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6d532470a8..8f89c4fe72 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1762,6 +1762,39 @@ s

[Qemu-devel] [PATCH v9 07/10] iscsi: Create and use iscsi_co_wait_for_task

2018-06-01 Thread Fam Zheng
This loop is repeated a growing number times. Make a helper. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- block/iscsi.c | 54 --- 1 file changed, 17 insertions(+), 37 deletions(-) diff --git a/block/iscsi.c

[Qemu-devel] [PATCH v9 09/10] block-backend: Add blk_co_copy_range

2018-06-01 Thread Fam Zheng
It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++ include/sysemu/block-backend.h | 4 2 files changed, 22 insertions(+) diff --git a/block/block-backend.c b/block/block

[Qemu-devel] [PATCH v9 03/10] raw: Implement copy offloading

2018-06-01 Thread Fam Zheng
Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 32 1 file changed, 32 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index b69a0674b3..f2e468df6f 100644 --- a/block/raw-forma

[Qemu-devel] [PATCH v9 06/10] iscsi: Query and save device designator when opening

2018-06-01 Thread Fam Zheng
The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c | 41

[Qemu-devel] [PATCH v9 05/10] file-posix: Implement bdrv_co_copy_range

2018-06-01 Thread Fam Zheng
With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng --- block/file-posix.c | 98 +++-- configure | 17 +++ include/block/raw-aio.h | 10 - 3 files changed, 120 insertions(+), 5

[Qemu-devel] [PATCH v9 02/10] raw: Check byte range uniformly

2018-06-01 Thread Fam Zheng
So there is no visible behavior change in adding the check code. The int64_t -> uint64_t inconsistency, as shown by the type casting, is pre-existing due to the interface. Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake Signed-off-by: Fam Zheng --- block/raw-format.

[Qemu-devel] [PATCH v9 01/10] block: Introduce API for copy offloading

2018-06-01 Thread Fam Zheng
copy_file_range(2). Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/io.c| 97 +++ include/block/block.h | 32 + include/block/block_int.h | 38 +++ 3 files changed, 167 insertions(+) diff --git

[Qemu-devel] [PATCH v9 00/10] qemu-img convert with copy offloading

2018-06-01 Thread Fam Zheng
offloading in scsi-disk (handle EXTENDED COPY command), and use the API in block jobs too. Fam Zheng (10): block: Introduce API for copy offloading raw: Check byte range uniformly raw: Implement copy offloading qcow2: Implement copy offloading file-posix: Implement bdrv_co_copy_range iscsi

Re: [Qemu-devel] [PATCH v8 00/11] qemu-img convert with copy offloading

2018-06-01 Thread Fam Zheng
On Thu, 05/31 23:45, no-re...@patchew.org wrote: > Hi, > > This series failed build test on s390x host. Please find the details below. > > Type: series > Message-id: 20180601062849.28641-1-f...@redhat.com > Subject: [Qemu-devel] [PATCH v8 00/11] qemu-img convert with copy offloading > > ===

[Qemu-devel] [PATCH] file-posix: Consolidate the locking error message

2018-06-01 Thread Fam Zheng
some change to consume that data. Before that is fully sorted out, let's just do the easy fix by joining the two lines. Signed-off-by: Fam Zheng --- block/file-posix.c | 10 ++-- tests/qemu-iotests/153.out | 99 +- tests/qemu-iotests/182.out | 3 +- 3

[Qemu-devel] [PATCH v8 10/11] block-backend: Add blk_co_copy_range

2018-06-01 Thread Fam Zheng
It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++ include/sysemu/block-backend.h | 4 2 files changed, 22 insertions(+) diff --git a/block/block-backend.c b/block/block

[Qemu-devel] [PATCH v8 11/11] qemu-img: Convert with copy offloading

2018-06-01 Thread Fam Zheng
The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Signed-off-by: Fam

[Qemu-devel] [PATCH v8 04/11] raw: Implement copy offloading

2018-06-01 Thread Fam Zheng
Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 32 1 file changed, 32 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index b69a0674b3..f2e468df6f 100644 --- a/block/raw-forma

[Qemu-devel] [PATCH v8 06/11] file-posix: Implement bdrv_co_copy_range

2018-06-01 Thread Fam Zheng
With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng --- block/file-posix.c | 98 +++-- configure | 17 +++ include/block/raw-aio.h | 10 - 3 files changed, 120 insertions(+), 5

[Qemu-devel] [PATCH v8 03/11] raw: Check byte range uniformly

2018-06-01 Thread Fam Zheng
So there is no visible behavior change in adding the check code. The int64_t -> uint64_t inconsistency, as shown by the type casting, is pre-existing due to the interface. Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake Signed-off-by: Fam Zheng --- block/raw-format.

[Qemu-devel] [PATCH v8 08/11] iscsi: Create and use iscsi_co_wait_for_task

2018-06-01 Thread Fam Zheng
This loop is repeated a growing number times. Make a helper. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- block/iscsi.c | 54 --- 1 file changed, 17 insertions(+), 37 deletions(-) diff --git a/block/iscsi.c

[Qemu-devel] [PATCH v8 05/11] qcow2: Implement copy offloading

2018-06-01 Thread Fam Zheng
igned-off-by: Fam Zheng --- block/qcow2.c | 229 +++--- 1 file changed, 199 insertions(+), 30 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 6d532470a8..8f89c4fe72 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -1762,6 +1762,39 @@ s

[Qemu-devel] [PATCH v8 02/11] block: Introduce API for copy offloading

2018-06-01 Thread Fam Zheng
copy_file_range(2). Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/io.c| 97 +++ include/block/block.h | 32 + include/block/block_int.h | 38 +++ 3 files changed, 167 insertions(+) diff --git

[Qemu-devel] [PATCH v8 07/11] iscsi: Query and save device designator when opening

2018-06-01 Thread Fam Zheng
The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c | 41

[Qemu-devel] [PATCH v8 09/11] iscsi: Implement copy offloading

2018-06-01 Thread Fam Zheng
Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219 +++ include/scsi

[Qemu-devel] [PATCH v8 01/11] docker: Update fedora image to 28

2018-06-01 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/docker/dockerfiles/fedora.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index b706f42405..65d7761cf5 100644 --- a/tests/docker/dockerfiles

[Qemu-devel] [PATCH v8 00/11] qemu-img convert with copy offloading

2018-06-01 Thread Fam Zheng
command), and use the API in block jobs too. Fam Zheng (11): docker: Update fedora image to 28 block: Introduce API for copy offloading raw: Check byte range uniformly raw: Implement copy offloading qcow2: Implement copy offloading file-posix: Implement bdrv_co_copy_range iscsi: Query

[Qemu-devel] [PATCH] docker: Update fedora image to 28

2018-05-31 Thread Fam Zheng
Signed-off-by: Fam Zheng --- tests/docker/dockerfiles/fedora.docker | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docker/dockerfiles/fedora.docker b/tests/docker/dockerfiles/fedora.docker index b706f42405..65d7761cf5 100644 --- a/tests/docker/dockerfiles

Re: [Qemu-devel] [PATCH v7 00/10] qemu-img convert with copy offloading

2018-05-31 Thread Fam Zheng
On Wed, 05/30 17:06, Stefan Hajnoczi wrote: > On Tue, May 29, 2018 at 01:59:49PM +0800, Fam Zheng wrote: > > v7: Fix qcow2. > > > > v6: Pick up rev-by from Stefan and Eric. > > Tweak patch 2 commit message. > > > > v5: - Fix raw offset/byt

[Qemu-devel] [PATCH 1/2] block: Honour BDRV_REQ_NO_SERIALISING in copy range

2018-05-30 Thread Fam Zheng
This semantics is needed by drive-backup so implement it before using this API there. Signed-off-by: Fam Zheng --- block/io.c| 6 -- include/block/block.h | 5 +++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/block/io.c b/block/io.c index b7beaeeb9f

[Qemu-devel] [PATCH 0/2] backup: Use copy offloading

2018-05-30 Thread Fam Zheng
pports it. Fam Zheng (2): block: Honour BDRV_REQ_NO_SERIALISING in copy range backup: Use copy offloading block/backup.c| 93 +-- block/io.c| 6 ++-- block/trace-events| 1 + include/block/block.h | 5 +-- 4 files c

[Qemu-devel] [PATCH 2/2] backup: Use copy offloading

2018-05-30 Thread Fam Zheng
The implementation is similar to the 'qemu-img convert'. In the beginning of the job, offloaded copy is attempted. If it fails, further I/O will go through the existing bounce buffer code path. Signed-off-by: Fam Zheng --- block/backup.c | 93

Re: [Qemu-devel] [PATCH v3 3/5] Acceptance tests: add quick VNC tests

2018-05-30 Thread Fam Zheng
On Wed, 05/30 13:57, Stefan Hajnoczi wrote: > On Tue, May 29, 2018 at 03:37:28PM -0400, Cleber Rosa wrote: > > This patch adds a few simple behavior tests for VNC. > > > > Signed-off-by: Cleber Rosa > > --- > > tests/acceptance/vnc.py | 60 + > > 1 file

[Qemu-devel] [PATCH v7 08/10] iscsi: Implement copy offloading

2018-05-29 Thread Fam Zheng
Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219 +++ include

<    1   2   3   4   5   6   7   8   9   10   >