[Qemu-block] [PATCH v5 10/10] qemu-img: Convert with copy offloading

The new blk_co_copy_range interface offers a more efficient way in the case of network based storage. Make use of it to allow faster convert operation. Since copy offloading cannot do zero detection ('-S') and compression (-c), only try it when these options are not used. Signed-off-by: Fam

[Qemu-block] [PATCH v5 09/10] block-backend: Add blk_co_copy_range

It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++ include/sysemu/block-backend.h | 4 2 files changed, 22 insertions(+) diff --git

[Qemu-block] [PATCH v5 08/10] iscsi: Implement copy offloading

Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219

[Qemu-block] [PATCH v5 06/10] iscsi: Query and save device designator when opening

The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi ---

[Qemu-block] [PATCH v5 04/10] qcow2: Implement copy offloading

The two callbacks are implemented quite similarly to the read/write functions: bdrv_co_copy_range_from maps for read and calls into bs->file or bs->backing depending on the allocation status; bdrv_co_copy_range_to maps for write and calls into bs->file. Signed-off-by: Fam Zheng

[Qemu-block] [PATCH v5 07/10] iscsi: Create and use iscsi_co_wait_for_task

This loop is repeated a growing number times. Make a helper. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi Reviewed-by: Eric Blake --- block/iscsi.c | 54 +- 1 file changed, 17

[Qemu-block] [PATCH v5 05/10] file-posix: Implement bdrv_co_copy_range

With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/file-posix.c | 96 +++-- include/block/raw-aio.h | 10 -- 2

[Qemu-block] [PATCH v5 03/10] raw: Implement copy offloading

Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 32 1 file changed, 32 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index

[Qemu-block] [PATCH v5 02/10] raw: Check byte range uniformly

We don't verify the request range against s->size in the I/O callbacks except for raw_co_pwritev. This is wrong (especially for raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them. Signed-off-by: Fam Zheng --- block/raw-format.c | 64

[Qemu-block] [PATCH v5 01/10] block: Introduce API for copy offloading

Introduce the bdrv_co_copy_range() API for copy offloading. Block drivers implementing this API support efficient copy operations that avoid reading each block from the source device and writing it to the destination devices. Examples of copy offload primitives are SCSI EXTENDED COPY and Linux

[Qemu-block] [PATCH v5 00/10] qemu-img convert with copy offloading

v5: - Fix raw offset/bytes check for read. [Eric] - Fix qcow2_handle_l2meta. [Stefan] - Add coroutine_fn whereever appropriate. [Stefan] v4: - Fix raw offset and size. [Eric] - iscsi: Drop unnecessary return values and variables in favor of constants. [Stefan] - qcow2:

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/3] qemu-iotests: Filter NFS paths

On Fri, 05/18 16:26, Kevin Wolf wrote: > NFS paths were only partially filtered in _filter_img_create, _img_info > and _filter_img_info, resulting in "nfs://127.0.0.1TEST_DIR/t.IMGFMT". > This adds another replacement to the sed calls that matches the test > directory not as a host path, but as an

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/3] qemu-iotests: 086 doesn't work with NFS

On Fri, 05/18 16:26, Kevin Wolf wrote: > The reference output file only works for file. 'qemu-img convert -p' > makes a lot more progress updates for NFS than for file, so disable the > test for NFS. > > Signed-off-by: Kevin Wolf Reviewed-by: Fam Zheng

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/3] qemu-iotests: Fix paths for NFS

On Fri, 05/18 16:26, Kevin Wolf wrote: > Test cases were trying to use nfs:// URLs as local filenames, which made > every test fail for NFS. With TEST_IMG and TEST_IMG_FILE set like for > the other protocols, NFS tests can pass again. > > Signed-off-by: Kevin Wolf Reviewed-by:

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] sheepdog: cleanup repeated expression

On Tue, 05/22 22:10, Paolo Bonzini wrote: > The expression "SD_INODE_SIZE - sizeof(inode.data_vdi_id)" already has a macro > defined for the same value (though with a nicer definition using offsetof). > Replace it. > > Signed-off-by: Paolo Bonzini Reviewed-by: Fam Zheng

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

On Tue, 05/22 17:02, Kevin Wolf wrote: > Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > > Hi, > > > > > > > You must /sometimes/ supply the correct machine type. > > > > > > > > It is quite dependent on the

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] sheepdog: remove huge BSS object

On Tue, 05/22 22:10, Paolo Bonzini wrote: > block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss > section. Replace it with a heap-allocated block, and make it smaller too > since only the inode header is actually being used. > > bss size goes down from 4464280 to 269976. >

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] sheepdog: cleanup repeated expression

On 05/22/2018 05:10 PM, Paolo Bonzini wrote: > The expression "SD_INODE_SIZE - sizeof(inode.data_vdi_id)" already has a macro > defined for the same value (though with a nicer definition using offsetof). > Replace it. > > Signed-off-by: Paolo Bonzini Reviewed-by: Philippe

[Qemu-block] [PATCH 2/2] sheepdog: remove huge BSS object

block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss section. Replace it with a heap-allocated block, and make it smaller too since only the inode header is actually being used. bss size goes down from 4464280 to 269976. Signed-off-by: Paolo Bonzini

[Qemu-block] [PATCH 0/2] sheepdog: remove huge BSS object

block/sheepdog.o has a 4M static variable that is 90% of QEMU's whole .bss section. Since it doesn't really have to be static, we can just use a heap allocated block. We can actually make it smaller too. :) Patch 1 is a related cleanup since we're touching that area of the code. Paolo Paolo

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/40] Generic background jobs

Hi On Tue, May 22, 2018 at 1:01 PM, Kevin Wolf wrote: > Am 18.05.2018 um 20:41 hat Dr. David Alan Gilbert geschrieben: >> * Kevin Wolf (kw...@redhat.com) wrote: >> > Before we can make x-blockdev-create a background job, we need to >> > generalise the job infrastructure so that

Re: [Qemu-block] [PATCH 3/4] nbd/client: Support requests of additional block sizing info

02.05.2018 00:13, Eric Blake wrote: The NBD spec is clarifying [1] that a server may want to advertise different limits for READ/WRITE (in our case, 32M) than for TRIM/ZERO (in our case, nearly 4G). Implement the client side support for these alternate limits, by always requesting the new

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

On Tue, May 22, 2018 at 05:02:21PM +0200, Kevin Wolf wrote: > Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > > Hi, > > > > > > > You must /sometimes/ supply the correct machine type. > > > > > > > > It is quite

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

Am 22.05.2018 um 16:19 hat Michael S. Tsirkin geschrieben: > On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > > Hi, > > > > > You must /sometimes/ supply the correct machine type. > > > > > > It is quite dependent on the guest OS you have installed, and even > > > just how the

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > Hi, > > > You must /sometimes/ supply the correct machine type. > > > > It is quite dependent on the guest OS you have installed, and even > > just how the guest OS is configured. In general Linux is very > > flexible and can

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/40] Generic background jobs

Am 18.05.2018 um 20:41 hat Dr. David Alan Gilbert geschrieben: > * Kevin Wolf (kw...@redhat.com) wrote: > > Before we can make x-blockdev-create a background job, we need to > > generalise the job infrastructure so that it can be used without any > > associated block node. > > Is there any

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

On Tue, May 22, 2018 at 09:35:55AM +0200, Gerd Hoffmann wrote: > Hi, > > > You must /sometimes/ supply the correct machine type. > > > > It is quite dependent on the guest OS you have installed, and even > > just how the guest OS is configured. In general Linux is very > > flexible and can

Re: [Qemu-block] [PATCH v2 37/40] job: Add query-jobs QMP command

Am 18.05.2018 um 20:22 hat Eric Blake geschrieben: > On 05/18/2018 08:21 AM, Kevin Wolf wrote: > > This adds a minimal query-jobs implementation that shouldn't pose many > > design questions. It can later be extended to expose more information, > > and especially job-specific information. > > > >

Re: [Qemu-block] [PATCH v2 36/40] job: Add lifecycle QMP commands

Am 18.05.2018 um 20:12 hat Eric Blake geschrieben: > On 05/18/2018 08:21 AM, Kevin Wolf wrote: > > This adds QMP commands that control the transition between states of the > > job lifecycle. > > > > Signed-off-by: Kevin Wolf > > --- > > qapi/job.json | 99

Re: [Qemu-block] [Qemu-devel] storing machine data in qcow images?

Hi, > You must /sometimes/ supply the correct machine type. > > It is quite dependent on the guest OS you have installed, and even > just how the guest OS is configured. In general Linux is very > flexible and can adapt to a wide range of hardware, automatically > detecting things as needed.