Re: [Qemu-block] [Qemu-devel] [RFC PATCH 00/12] qemu-img: add bitmap queries

2018-05-11 Thread no-reply
Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20180512012537.22478-1-js...@redhat.com Subject: [Qemu-devel] [RFC PATCH 00/12] qemu-img: add bitmap queries === TEST SCRIPT BEGIN === #!/bin/bash BASE=base n=1 total=$(git

[Qemu-block] [RFC PATCH 12/12] qemu-img: add bitmap clear

2018-05-11 Thread John Snow
Signed-off-by: John Snow --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 46 -- 2 files changed, 46 insertions(+), 4 deletions(-) diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx index d25f359f5a..7b6ec73488 100644 ---

[Qemu-block] [RFC PATCH 10/12] qemu-img: split off common chunk of map command

2018-05-11 Thread John Snow
It will be re-used for a bitmap listing command. Signed-off-by: John Snow --- qemu-img.c | 192 +++-- 1 file changed, 110 insertions(+), 82 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index ea62d2d61e..e31e38f674

[Qemu-block] [RFC PATCH 06/12] qapi: add bitmap info

2018-05-11 Thread John Snow
Add some of the necessary scaffolding for reporting bitmap information. Signed-off-by: John Snow --- qapi/block-core.json | 60 +++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json

[Qemu-block] [RFC PATCH 04/12] qcow2/dirty-bitmaps: load IN_USE bitmaps if disk is RO

2018-05-11 Thread John Snow
For the purposes of qemu-img manipulation and querying of bitmaps, load bitmaps that are "in use" -- if the image is read only. This will allow us to diagnose problems with this flag using the qemu-img tool. Signed-off-by: John Snow --- block/qcow2-bitmap.c | 32

[Qemu-block] [RFC PATCH 01/12] qcow2-bitmap: cache bm_list

2018-05-11 Thread John Snow
We don't need to re-read this list every time, exactly. We can keep it cached and delete our copy when we flush to disk. Because we don't try to flush bitmaps on close if there's nothing to flush, add a new conditional to delete the state anyway for a clean exit. Signed-off-by: John Snow

[Qemu-block] [RFC PATCH 09/12] qapi/block-core: add BitmapMapping and BitmapEntry structs

2018-05-11 Thread John Snow
Add two new structures for detailing the marked regions of bitmaps as saved in e.g. qcow2 files. Signed-off-by: John Snow --- qapi/block-core.json | 32 1 file changed, 32 insertions(+) diff --git a/qapi/block-core.json b/qapi/block-core.json

[Qemu-block] [RFC PATCH 02/12] qcow2/dirty-bitmap: cache loaded bitmaps

2018-05-11 Thread John Snow
For bitmaps that we succeeded in loading, we can cache a reference to that object. This will let us iterate over the more convenient form of in-memory bitmaps for qemu-img bitmap manipulation tools. Signed-off-by: John Snow --- block/qcow2-bitmap.c | 1 + 1 file changed, 1

[Qemu-block] [RFC PATCH 00/12] qemu-img: add bitmap queries

2018-05-11 Thread John Snow
Allow qemu-img to show information about bitmaps stored in qcow2 images. Add a 'bitmap' meta-command with 'dump' sub-command to retrieve a list of dirty regions in bitmaps stored in a qcow2 image. RFC: - I am not 1000% convinced the bm_list caching is perfectly safe, especially with respect to

[Qemu-block] [RFC PATCH 05/12] qcow2-bitmap: track bitmap type

2018-05-11 Thread John Snow
We only have one type of persistent bitmap right now, but I'd like the qemu-img tool to be able to give good diagnostic information if it sees an unknown/unsupported type. We do enforce it to be the dirty tracking type in check_dir_entry, but I wanted positive affirmation of the type in the

[Qemu-block] [RFC PATCH 08/12] qjson: allow caller to ask for arbitrary indent

2018-05-11 Thread John Snow
The function already exists, just expose it. Signed-off-by: John Snow --- include/qapi/qmp/qjson.h | 1 + qobject/qjson.c | 21 +++-- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/include/qapi/qmp/qjson.h b/include/qapi/qmp/qjson.h

[Qemu-block] [RFC PATCH 07/12] qcow2-bitmap: add basic bitmaps info

2018-05-11 Thread John Snow
Add functions for querying the basic information inside of bitmaps. Restructure the bitmaps flags masks to facilitate providing a list of flags belonging to the bitmap(s) being queried. Signed-off-by: John Snow --- block/qcow2-bitmap.c | 81

[Qemu-block] [RFC PATCH 03/12] block/qcow2-bitmap: avoid adjusting bm->flags for RO bitmaps

2018-05-11 Thread John Snow
Instead of always setting IN_USE, handle whether or not the bitmap is read-only instead of a two-loop pass. This will allow us to show the flags exactly as they appear for bitmaps in `qemu-img info`. Signed-off-by: John Snow --- block/qcow2-bitmap.c | 48

Re: [Qemu-block] [PATCH 11/42] job: Add job_delete()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > This moves freeing the Job object and its fields from block_job_unref() > to job_delete(). > > Signed-off-by: Kevin Wolf > --- > include/qemu/job.h | 3 +++ > blockjob.c | 3 +-- > job.c | 6 ++ > 3 files

Re: [Qemu-block] [PATCH 10/42] job: Add JobDriver.job_type

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > This moves the job_type field from BlockJobDriver to JobDriver. > > Signed-off-by: Kevin Wolf > --- > include/block/blockjob_int.h | 3 --- > include/qemu/job.h | 11 +++ > block/backup.c | 2 +- >

Re: [Qemu-block] [PATCH 09/42] job: Rename BlockJobType into JobType

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > QAPI types aren't externally visible, so we can rename them without > causing problems. Before we add a job type to Job, rename the enum > so it can be used for more than just block jobs. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric

Re: [Qemu-block] [PATCH 08/42] job: Create Job, JobDriver and job_create()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > This is the first step towards creating an infrastructure for generic > background jobs that aren't tied to a block device. For now, Job only > stores its ID and JobDriver, the rest stays in BlockJob. > > The following patches will move over more parts of

Re: [Qemu-block] [Qemu-devel] [RFC 00/10] [TESTING NEEDED] python: futurize --stage1 (Python 3 compatibility)

2018-05-11 Thread no-reply
Hi, This series seems to have some coding style problems. See output below for more information: Type: series Message-id: 20180511222052.8734-1-ehabk...@redhat.com Subject: [Qemu-devel] [RFC 00/10] [TESTING NEEDED] python: futurize --stage1 (Python 3 compatibility) === TEST SCRIPT BEGIN ===

Re: [Qemu-block] [PATCH 06/42] blockjob: Add block_job_driver()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > The backup block job directly accesses the driver field in BlockJob. Add > a wrapper for getting it. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric Blake > --- > include/block/blockjob.h | 7 +++ >

Re: [Qemu-block] [PATCH 07/42] blockjob: Remove block_job_pause/resume_all()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > Commit 81193349 removed the only use of block_job_pause/resume_all(), > which was in bdrv_drain_all(). The functions are now unused and can be > removed. I have a strange liking for all-digit commit hash prefixes. > Signed-off-by: Kevin Wolf

Re: [Qemu-block] [Qemu-devel] [PATCH 01/42] blockjob: Fix assertion in block_job_finalize()

2018-05-11 Thread John Snow
On 05/09/2018 12:25 PM, Kevin Wolf wrote: > Every job gets a non-NULL job->txn on creation, but it doesn't > necessarily keep it until it is decommissioned: Finalising a job removes > it from its transaction. Therefore, calling 'blockdev-job-finalize' a > second time on an already concluded job

[Qemu-block] [RFC 10/10] python: futurize -f lib2to3.fixes.fix_numliterals

2018-05-11 Thread Eduardo Habkost
Convert octal literals into the new syntax. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_numliterals $py Signed-off-by: Eduardo

[Qemu-block] [RFC 06/10] python: futurize -f lib2to3.fixes.fix_reduce

2018-05-11 Thread Eduardo Habkost
Handle the move of reduce() to functools.reduce(). This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_reduce $py Signed-off-by: Eduardo

[Qemu-block] [RFC 05/10] python: futurize -f lib2to3.fixes.fix_standarderror

2018-05-11 Thread Eduardo Habkost
Rename StandardError to Exception. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_standarderror $py Signed-off-by: Eduardo Habkost

[Qemu-block] [RFC 09/10] python: futurize -f lib2to3.fixes.fix_except

2018-05-11 Thread Eduardo Habkost
Convert "except X, T" to "except X as T". This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_except $py Signed-off-by: Eduardo Habkost

Re: [Qemu-block] [PATCH 05/42] blockjob: Introduce block_job_ratelimit_get_delay()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:26, Kevin Wolf wrote: > This gets us rid of more direct accesses to BlockJob fields from the > job drivers. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric Blake > --- > include/block/blockjob_int.h | 8 > block/backup.c

[Qemu-block] [RFC 07/10] python: futurize -f lib2to3.fixes.fix_tuple_params

2018-05-11 Thread Eduardo Habkost
Remove implicit tuple parameter unpacking. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_tuple_params $py Signed-off-by: Eduardo

[Qemu-block] [RFC 08/10] python: futurize -f lib2to3.fixes.fix_renames

2018-05-11 Thread Eduardo Habkost
Change sys.maxint to sys.maxsize. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_renames $py Signed-off-by: Eduardo Habkost

[Qemu-block] [RFC 01/10] python: futurize -f libfuturize.fixes.fix_print_with_import

2018-05-11 Thread Eduardo Habkost
Change all Python code to use print as a function. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f libfuturize.fixes.fix_print_with_import $py

[Qemu-block] [RFC 04/10] python: futurize -f lib2to3.fixes.fix_has_key

2018-05-11 Thread Eduardo Habkost
Change "dict.has_key(key)" to "key in dict" This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f lib2to3.fixes.fix_has_key $py Signed-off-by: Eduardo Habkost

[Qemu-block] [RFC 02/10] python: futurize -f libfuturize.fixes.fix_absolute_import

2018-05-11 Thread Eduardo Habkost
Make implicit relative imports explicit and add "from __future__ import absolute_import" at the top of each relevant module. This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4)

[Qemu-block] [RFC 03/10] python: futurize -f libfuturize.fixes.fix_next_call

2018-05-11 Thread Eduardo Habkost
Change obj.next() calls to next(obj). This is necessary for Python 3 compatibility. Done using: $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \ sort -u | grep -v README.sh4) $ futurize -w -f libfuturize.fixes.fix_next_call $py Signed-off-by: Eduardo Habkost

[Qemu-block] [RFC 00/10] [TESTING NEEDED] python: futurize --stage1 (Python 3 compatibility)

2018-05-11 Thread Eduardo Habkost
TESTING NEEDED: Due to the amount of changes, I didn't test all scripts touched by this series. If you are responsible for any of the touched files, I would appreciate help on testing the series. >From the futurize[1] documentation: > This applies fixes that modernize Python 2 code without >

Re: [Qemu-block] [PATCH 04/42] blockjob: Implement block_job_set_speed() centrally

2018-05-11 Thread Max Reitz
On 2018-05-09 18:25, Kevin Wolf wrote: > All block job drivers support .set_speed and all of them duplicate the > same code to implement it. Move that code to blockjob.c and remove the > now useless callback. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric Blake

Re: [Qemu-block] [PATCH 03/42] blockjob: Move RateLimit to BlockJob

2018-05-11 Thread Max Reitz
On 2018-05-09 18:25, Kevin Wolf wrote: > Every block job has a RateLimit, and they all do the exact same thing > with it, so it should be common infrastructure. Move the struct field > for a start. > > Signed-off-by: Kevin Wolf > Reviewed-by: Eric Blake >

Re: [Qemu-block] [PATCH 02/42] blockjob: Wrappers for progress counter access

2018-05-11 Thread Max Reitz
On 2018-05-09 18:25, Kevin Wolf wrote: > Block job drivers are not expected to mess with the internals of the > BlockJob object, so provide wrapper functions for one of the cases where > they still do it: Updating the progress counter. > > Signed-off-by: Kevin Wolf >

Re: [Qemu-block] [PATCH 01/42] blockjob: Fix assertion in block_job_finalize()

2018-05-11 Thread Max Reitz
On 2018-05-09 18:25, Kevin Wolf wrote: > Every job gets a non-NULL job->txn on creation, but it doesn't > necessarily keep it until it is decommissioned: Finalising a job removes > it from its transaction. Therefore, calling 'blockdev-job-finalize' a > second time on an already concluded job

Re: [Qemu-block] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval()

2018-05-11 Thread Max Reitz
On 2018-05-11 20:39, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> The purpose of this function is to prepare a QDict for consumption by >> the keyval visitor, which only accepts strings and QNull. >> >> Signed-off-by: Max Reitz >> --- >>  

Re: [Qemu-block] [PATCH 10/13] tests: Add QDict clone-flatten test

2018-05-11 Thread Max Reitz
On 2018-05-11 20:46, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> This new test verifies that qdict_flatten() does not modify a shallow >> clone of the given QDict. >> >> Signed-off-by: Max Reitz >> --- >>   tests/check-qdict.c | 33

Re: [Qemu-block] [Qemu-devel] Restoring bitmaps after failed/cancelled migration

2018-05-11 Thread John Snow
On 04/18/2018 10:00 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all. > > We now have the following problem: > > If dirty-bitmaps migration capability is enabled, persistance flag is > dropped for all migrated bitmaps, to prevent their storing to the > storage on inactivate. It works ok,

Re: [Qemu-block] [PATCH 10/13] tests: Add QDict clone-flatten test

2018-05-11 Thread Eric Blake
On 05/09/2018 11:55 AM, Max Reitz wrote: This new test verifies that qdict_flatten() does not modify a shallow clone of the given QDict. Signed-off-by: Max Reitz --- tests/check-qdict.c | 33 + 1 file changed, 33 insertions(+) I'm not

Re: [Qemu-block] [PATCH 09/13] qdict: Make qdict_flatten() shallow-clone-friendly

2018-05-11 Thread Eric Blake
On 05/09/2018 11:55 AM, Max Reitz wrote: In its current form, qdict_flatten() removes all entries from nested QDicts that are moved to the root QDict. It is completely sufficient to remove all old entries from the root QDict, however. If the nested dicts have a refcount of 1, this will

Re: [Qemu-block] [PATCH 07/13] qdict: Add qdict_stringify_for_keyval()

2018-05-11 Thread Eric Blake
On 05/09/2018 11:55 AM, Max Reitz wrote: The purpose of this function is to prepare a QDict for consumption by the keyval visitor, which only accepts strings and QNull. Signed-off-by: Max Reitz --- include/block/qdict.h | 2 ++ qobject/qdict.c | 57

Re: [Qemu-block] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test

2018-05-11 Thread Eric Blake
On 05/11/2018 01:13 PM, Max Reitz wrote: + * "h": 0x, + * "i": true, + * "j": 0 Is it worth testing fun things like '-0.0'? Sure, why not. Maybe even infinity, although I'm not quite sure the input visitor can handle it... JSON can't handle Inf or

Re: [Qemu-block] [PATCH 01/13] qapi: Add default-variant for flat unions

2018-05-11 Thread Eric Blake
On 05/11/2018 12:59 PM, Max Reitz wrote: On 2018-05-10 15:18, Eric Blake wrote: On 05/10/2018 08:12 AM, Eric Blake wrote: Oh, I just had a thought: +++ b/scripts/qapi/visit.py @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members,   if variants: +    if

Re: [Qemu-block] [PATCH 08/13] tests: Add qdict_stringify_for_keyval() test

2018-05-11 Thread Max Reitz
On 2018-05-10 18:02, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >>   tests/check-qdict.c | 54 >> + >>   1 file changed, 54 insertions(+) >> > >> +static void

Re: [Qemu-block] [PATCH 06/13] block: Add block-specific QDict header

2018-05-11 Thread Max Reitz
On 2018-05-10 16:54, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> There are numerous QDict functions that have been introduced for and are >> used only by the block layer.  Move their declarations into an own > > s/an own/their own/ > >> header file to reflect that. >> >>

Re: [Qemu-block] [PATCH 05/13] qapi: Formalize qcow encryption probing

2018-05-11 Thread Max Reitz
On 2018-05-10 16:24, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> Currently, you can give no encryption format for a qcow file while still >> passing a key-secret.  That does not conform to the schema, so this >> patch changes the schema to allow it. >> >> Signed-off-by: Max

Re: [Qemu-block] [PATCH 03/13] tests: Add QAPI optional discriminator tests

2018-05-11 Thread Max Reitz
On 2018-05-10 16:08, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> There already is an optional discriminator test, although it also noted >> the discriminator name itself as optional.  Changing that, and adding a >> default-variant field, makes that test pass instead of failing.

Re: [Qemu-block] [PATCH 01/13] qapi: Add default-variant for flat unions

2018-05-11 Thread Max Reitz
On 2018-05-10 15:18, Eric Blake wrote: > On 05/10/2018 08:12 AM, Eric Blake wrote: > > Oh, I just had a thought: > >>> +++ b/scripts/qapi/visit.py >>> @@ -40,10 +40,20 @@ def gen_visit_object_members(name, base, members, > >>>   if variants: >>> +    if variants.default_tag_value is

Re: [Qemu-block] [PATCH 01/13] qapi: Add default-variant for flat unions

2018-05-11 Thread Max Reitz
On 2018-05-10 15:12, Eric Blake wrote: > On 05/09/2018 11:55 AM, Max Reitz wrote: >> This patch allows specifying a discriminator that is an optional member >> of the base struct.  In such a case, a default value must be provided >> that is used when no value is given. >> >> Signed-off-by: Max

Re: [Qemu-block] [Qemu-devel] [PATCH 04/13] qapi: Formalize qcow2 encryption probing

2018-05-11 Thread Max Reitz
On 2018-05-10 09:58, Daniel P. Berrangé wrote: > On Wed, May 09, 2018 at 06:55:21PM +0200, Max Reitz wrote: >> Currently, you can give no encryption format for a qcow2 file while >> still passing a key-secret. That does not conform to the schema, so >> this patch changes the schema to allow it.

Re: [Qemu-block] [PATCH v3] qcow2: fix preallocation with metadata on bare block device

2018-05-11 Thread Kevin Wolf
Am 11.05.2018 um 17:36 hat Ivan Ren geschrieben: > Create a qcow2 directly on bare block device with > "-o preallocation=metadata" option. When read this qcow2, it will > return pre-existing data on block device. This patch add > QCOW_OFLAG_ZERO flag (supported in qcow_version >= 3) for >

Re: [Qemu-block] [Qemu-devel] Some question about savem/qcow2 incremental snapshot

2018-05-11 Thread Kevin Wolf
Am 10.05.2018 um 10:26 hat Stefan Hajnoczi geschrieben: > On Wed, May 09, 2018 at 07:54:31PM +0200, Max Reitz wrote: > > On 2018-05-09 12:16, Stefan Hajnoczi wrote: > > > On Tue, May 08, 2018 at 05:03:09PM +0200, Kevin Wolf wrote: > > >> Am 08.05.2018 um 16:41 hat Eric Blake geschrieben: > > >>>

Re: [Qemu-block] [PATCH v2 0/2] block/file-posix: allow -drive cache.direct=off live migration

2018-05-11 Thread Stefan Hajnoczi
On Fri, Apr 27, 2018 at 05:23:10PM +0100, Stefan Hajnoczi wrote: > v2: > * Add comment on !__linux__ situation [Fam] > * Add file-posix.c x-check-cache-dropped=on|off option [DaveG, Kevin] > > file-posix.c only supports shared storage live migration with -drive > cache.direct=off due to cache

[Qemu-block] [PATCH v3] qcow2: fix preallocation with metadata on bare block device

2018-05-11 Thread Ivan Ren
Create a qcow2 directly on bare block device with "-o preallocation=metadata" option. When read this qcow2, it will return pre-existing data on block device. This patch add QCOW_OFLAG_ZERO flag (supported in qcow_version >= 3) for preallocated l2 entry to avoid this problem. Signed-off-by: Ivan

Re: [Qemu-block] [PATCH v4 02/10] raw: Check byte range uniformly

2018-05-11 Thread Eric Blake
On 05/11/2018 07:08 AM, Fam Zheng wrote: We don't verify the request range against s->size in the I/O callbacks except for raw_co_pwritev. This is wrong (especially for raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them. Did you bother identifying how long the bug has been present (but

Re: [Qemu-block] [Qemu-devel] [PATCH v2] qcow2: fix preallocation with metadata on bare block device

2018-05-11 Thread Eric Blake
On 05/11/2018 07:37 AM, Ivan Ren wrote: Create a qcow2 directly on bare block device with "-o preallocation=metadata" option. When read this qcow2, it will return pre-existing data on block device, and this may lead to data leakage. This patch add QCOW_OFLAG_ZERO for all preallocated l2 entry to

[Qemu-block] [PATCH v4 08/10] iscsi: Implement copy offloading

2018-05-11 Thread Fam Zheng
Issue EXTENDED COPY (LID1) command to implement the copy_range API. The parameter data construction code is modified from libiscsi's iscsi-dd.c. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/iscsi.c| 219

[Qemu-block] [PATCH v4 09/10] block-backend: Add blk_co_copy_range

2018-05-11 Thread Fam Zheng
It's a BlockBackend wrapper of the BDS interface. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/block-backend.c | 18 ++ include/sysemu/block-backend.h | 4 2 files changed, 22 insertions(+) diff --git

[Qemu-block] [PATCH v4 06/10] iscsi: Query and save device designator when opening

2018-05-11 Thread Fam Zheng
The device designator data returned in INQUIRY command will be useful to fill in source/target fields during copy offloading. Do this when connecting to the target and save the data for later use. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi ---

[Qemu-block] [PATCH v4 03/10] raw: Implement copy offloading

2018-05-11 Thread Fam Zheng
Just pass down to ->file. Signed-off-by: Fam Zheng Reviewed-by: Stefan Hajnoczi --- block/raw-format.c | 32 1 file changed, 32 insertions(+) diff --git a/block/raw-format.c b/block/raw-format.c index

[Qemu-block] [PATCH v4 05/10] file-posix: Implement bdrv_co_copy_range

2018-05-11 Thread Fam Zheng
With copy_file_range(2), we can implement the bdrv_co_copy_range semantics. Signed-off-by: Fam Zheng --- block/file-posix.c | 96 +++-- include/block/raw-aio.h | 10 -- 2 files changed, 101 insertions(+), 5 deletions(-) diff

[Qemu-block] [PATCH v4 02/10] raw: Check byte range uniformly

2018-05-11 Thread Fam Zheng
We don't verify the request range against s->size in the I/O callbacks except for raw_co_pwritev. This is wrong (especially for raw_co_pwrite_zeroes and raw_co_pdiscard), so fix them. Signed-off-by: Fam Zheng --- block/raw-format.c | 63

[Qemu-block] [PATCH v4 00/10] qemu-img convert with copy offloading

2018-05-11 Thread Fam Zheng
v4: - Fix raw offset and size. [Eric] - iscsi: Drop unnecessary return values and variables in favor of constants. [Stefan] - qcow2: Handle small backing case. [Stefan] - file-posix: Translate ENOSYS to ENOTSUP. [Stefan] - API documentation and commit message. [Stefan] -

[Qemu-block] [PATCH v4 01/10] block: Introduce API for copy offloading

2018-05-11 Thread Fam Zheng
Introduce the bdrv_co_copy_range() API for copy offloading. Block drivers implementing this API support efficient copy operations that avoid reading each block from the source device and writing it to the destination devices. Examples of copy offload primitives are SCSI EXTENDED COPY and Linux

Re: [Qemu-block] [Qemu-devel] [PATCH] qcow2: fix preallocation with metadata on bare block device

2018-05-11 Thread Ivan Ren
> This says what the patch does, but not why. What is the actual use case > scenario where changing semantics to have the qcow2 overwrite the > garbage to read 0 instead of any pre-existing garbage, when dealing with > portions of the disk that have not yet been written by the guest? Are > you

Re: [Qemu-block] [PATCH v3 4/9] file-posix: Implement bdrv_co_copy_range

2018-05-11 Thread Fam Zheng
On Thu, 05/10 09:50, Stefan Hajnoczi wrote: > On Wed, May 09, 2018 at 10:58:10PM +0800, Fam Zheng wrote: > > +static off_t copy_file_range(int in_fd, off_t *in_off, int out_fd, > > + off_t *out_off, size_t len, unsigned int > > flags) > > +{ > > +#ifdef

Re: [Qemu-block] [Qemu-devel] [PATCH v3 2/9] raw: Implement copy offloading

2018-05-11 Thread Fam Zheng
On Wed, 05/09 10:17, Eric Blake wrote: > On 05/09/2018 09:58 AM, Fam Zheng wrote: > > Just pass down to ->file. > > > > Signed-off-by: Fam Zheng > > Reviewed-by: Stefan Hajnoczi > > --- > > block/raw-format.c | 20 > > 1 file