Re: [Qemu-block] [PATCH v2 5/5] block/rbd: add support for 'mon_host', 'auth_supported' via QAPI

2017-02-28 Thread Daniel P. Berrange
On Mon, Feb 27, 2017 at 10:57:44PM -0500, Jeff Cody wrote: > On Mon, Feb 27, 2017 at 04:47:54PM -0600, Eric Blake wrote: > > On 02/27/2017 12:58 PM, Jeff Cody wrote: > > > @@ -604,6 +620,29 @@ static int qemu_rbd_open(BlockDriverState *bs, QDict > > > *options, int flags, > > > goto faile

Re: [Qemu-block] [PATCH v2 5/5] block/rbd: add support for 'mon_host', 'auth_supported' via QAPI

2017-02-27 Thread Daniel P. Berrange
On Mon, Feb 27, 2017 at 01:58:48PM -0500, Jeff Cody wrote: > This adds support for two additional options that may be specified > by QAPI in blockdev-add: > > mon_host: servername and port > auth_supported: either 'cephx' or 'none' > > Signed-off-by: Jeff Cody > --- > block/rbd.c

Re: [Qemu-block] [PATCH 4/4] block/rbd: Add blockdev-add support

2017-02-27 Thread Daniel P. Berrange
On Mon, Feb 27, 2017 at 02:30:41AM -0500, Jeff Cody wrote: > Signed-off-by: Jeff Cody > --- > qapi/block-core.json | 47 --- > 1 file changed, 44 insertions(+), 3 deletions(-) > > diff --git a/qapi/block-core.json b/qapi/block-core.json > index 5f82d35

Re: [Qemu-block] [PATCH 4/4] block/rbd: Add blockdev-add support

2017-02-27 Thread Daniel P. Berrange
On Mon, Feb 27, 2017 at 08:18:59AM -0500, Jeff Cody wrote: > On Mon, Feb 27, 2017 at 09:31:21AM +0000, Daniel P. Berrange wrote: > > On Mon, Feb 27, 2017 at 02:36:13AM -0500, Jeff Cody wrote: > > > On Mon, Feb 27, 2017 at 02:30:41AM -0500, Jeff Cody wrote: > > >

Re: [Qemu-block] [PATCH 2/4] block/rbd: code movement

2017-02-27 Thread Daniel P. Berrange
Describing this as "code movement" when the added & removed chunks are not identical is a bit misleading. Can you expand the commit message to explain why the extra options are being added On Mon, Feb 27, 2017 at 02:30:39AM -0500, Jeff Cody wrote: > Signed-off-by: Jeff Cody > --- > block/rbd.c

Re: [Qemu-block] [PATCH 4/4] block/rbd: Add blockdev-add support

2017-02-27 Thread Daniel P. Berrange
On Mon, Feb 27, 2017 at 02:36:13AM -0500, Jeff Cody wrote: > On Mon, Feb 27, 2017 at 02:30:41AM -0500, Jeff Cody wrote: > > Signed-off-by: Jeff Cody > > --- > > qapi/block-core.json | 47 --- > > 1 file changed, 44 insertions(+), 3 deletions(-) > > > >

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-24 Thread Daniel P. Berrange
On Fri, Feb 24, 2017 at 05:04:34PM +0100, Markus Armbruster wrote: > Markus Armbruster writes: > > [...] > > === Dotted keys === > > > > One sufficiently powerful syntax extension already exists: the dotted > > key convention. It's syntactically unambiguous only when none of the > > KEYs involve

Re: [Qemu-block] [PATCH v5 02/18] block: add ability to set a prefix for opt names

2017-02-23 Thread Daniel P. Berrange
On Thu, Feb 23, 2017 at 11:28:39AM +0100, Kevin Wolf wrote: > Am 22.02.2017 um 19:28 hat Eric Blake geschrieben: > > Using '.' would mean a layer of {} nesting on the wire, maybe as in: > > > > { "driver": "qcow2", ..., "luks" : { "hash-alg": ... } } > > > > but conceptually, I like that a bit be

Re: [Qemu-block] [PATCH v5 02/18] block: add ability to set a prefix for opt names

2017-02-22 Thread Daniel P. Berrange
On Wed, Feb 22, 2017 at 04:18:33PM +0100, Kevin Wolf wrote: > Am 21.02.2017 um 12:54 hat Daniel P. Berrange geschrieben: > > When integrating the crypto support with qcow/qcow2, we don't > > want to use the bare LUKS option names "hash-alg", "key-secret",

Re: [Qemu-block] [PATCH v3 4/4] qemu-img: copy *key-secret opts when opening newly created files

2017-02-22 Thread Daniel P. Berrange
On Wed, Feb 22, 2017 at 01:18:49PM +0100, Kevin Wolf wrote: > Am 22.02.2017 um 12:33 hat Daniel P. Berrange geschrieben: > > On Wed, Feb 22, 2017 at 12:20:36PM +0100, Kevin Wolf wrote: > > > Am 20.02.2017 um 16:19 hat Daniel P. Berrange geschrieben: > > > > The qem

Re: [Qemu-block] [PATCH v3 4/4] qemu-img: copy *key-secret opts when opening newly created files

2017-02-22 Thread Daniel P. Berrange
On Wed, Feb 22, 2017 at 12:20:36PM +0100, Kevin Wolf wrote: > Am 20.02.2017 um 16:19 hat Daniel P. Berrange geschrieben: > > The qemu-img dd/convert commands will create a image file and > > then try to open it. Historically it has been possible to open > > new files withou

Re: [Qemu-block] [PATCH v3 2/4] qemu-img: fix --image-opts usage with dd command

2017-02-22 Thread Daniel P. Berrange
On Wed, Feb 22, 2017 at 11:46:06AM +0100, Kevin Wolf wrote: > Am 20.02.2017 um 16:19 hat Daniel P. Berrange geschrieben: > > The --image-opts flag can only be used to affect the parsing > > of the source image. The target image has to be specified in > > the traditional style

[Qemu-block] [PATCH v5 16/18] block: rip out all traces of password prompting

2017-02-21 Thread Daniel P. Berrange
Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- h

[Qemu-block] [PATCH v5 17/18] block: remove all encryption handling APIs

2017-02-21 Thread Daniel P. Berrange
whether an image is encrypted or not, since that is a potentially useful piece of metadata to report to the user. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block.c | 77 +-- block/crypto.c| 1 - block

[Qemu-block] [PATCH v5 18/18] block: pass option prefix down to crypto layer

2017-02-21 Thread Daniel P. Berrange
to layer can report accurate error messages, we must tell it what option name prefix was used. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/crypto.c| 4 ++-- block/qcow.c | 7 --- block/qcow2.c | 15 +-- crypto/block-luks.c

[Qemu-block] [PATCH v5 13/18] qcow2: add support for LUKS encryption format

2017-02-21 Thread Daniel P. Berrange
sectors when qcow2 internal snapshots are used, thus giving stronger protection against watermarking attacks. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 4 +- block/qcow2-refcount.c | 10 ++ block/qcow2.c | 289 +++

[Qemu-block] [PATCH v5 10/18] qcow2: make qcow2_encrypt_sectors encrypt in place

2017-02-21 Thread Daniel P. Berrange
: Alberto Garcia Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 17 ++--- block/qcow2.c | 4 ++-- block/qcow2.h | 3 +-- 3 files changed, 9 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 928c1e2..907e869 100644

[Qemu-block] [PATCH v5 15/18] iotests: enable tests 134 and 158 to work with qcow (v1)

2017-02-21 Thread Daniel P. Berrange
The 138 and 158 iotests exercise the legacy qcow2 aes encryption code path and they work fine with qcow v1 too. Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/158 | 2 +- 2 files changed, 2 insertions

[Qemu-block] [PATCH v5 12/18] qcow2: extend specification to cover LUKS encryption

2017-02-21 Thread Daniel P. Berrange
Update the qcow2 specification to describe how the LUKS header is placed inside a qcow2 file, when using LUKS encryption for the qcow2 payload instead of the legacy AES-CBC encryption Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- docs/specs/qcow2.txt | 96

[Qemu-block] [PATCH v5 07/18] iotests: fix 097 when run with qcow

2017-02-21 Thread Daniel P. Berrange
ff-by: Daniel P. Berrange --- tests/qemu-iotests/097 | 10 +--- tests/qemu-iotests/097.out | 125 ++-- tests/qemu-iotests/175 | 126 + tests/qemu-iotests/175.out

[Qemu-block] [PATCH v5 08/18] qcow: make encrypt_sectors encrypt in place

2017-02-21 Thread Daniel P. Berrange
Garcia Reviewed-by: Eric Blake Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow.c | 44 +++- 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/block/qcow.c b/block/qcow.c index 744b25e..f07cdbb 100644 --- a/block/qcow.c

[Qemu-block] [PATCH v5 09/18] qcow: convert QCow to use QCryptoBlock for encryption

2017-02-21 Thread Daniel P. Berrange
: Daniel P. Berrange --- block/crypto.c | 10 +++ block/crypto.h | 9 +++ block/qcow.c | 184 +++ qapi/block-core.json | 17 - 4 files changed, 117 insertions(+), 103 deletions(-) diff --git a/block/crypto.c b/block/crypto.c

[Qemu-block] [PATCH v5 06/18] iotests: skip 048 with qcow which doesn't support resize

2017-02-21 Thread Daniel P. Berrange
Test 048 is designed to verify data preservation during an image resize. The qcow (v1) format impl has never supported resize so always fails. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/048 | 2 +- 1 file changed, 1 insertion

[Qemu-block] [PATCH v5 14/18] qcow2: add iotests to cover LUKS encryption support

2017-02-21 Thread Daniel P. Berrange
This extends the 087 iotest to cover LUKS encryption when doing blockdev-add. Two further tests are added to validate read/write of LUKS encrypted images with a single file and with a backing file. Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests

[Qemu-block] [PATCH v5 11/18] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-02-21 Thread Daniel P. Berrange
CPU state. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 47 +-- block/qcow2.c | 191 + block/qcow2.h | 5 +- qapi/block-core.json | 8 +- tests/qemu-iotests/049

[Qemu-block] [PATCH v5 05/18] iotests: skip 042 with qcow which dosn't support zero sized images

2017-02-21 Thread Daniel P. Berrange
Test 042 is designed to verify operation with zero sized images. Such images are not supported with qcow (v1), so this test has always failed. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/042 | 2 +- 1 file changed, 1 insertion

[Qemu-block] [PATCH v5 00/18] Convert QCow[2] to QCryptoBlock & add LUKS support

2017-02-21 Thread Daniel P. Berrange
tion-format' option - Deprecate legacy 'encryption' option - Drop redundant test scenarios - Use small file sizes for iotests - Drop pbkdf iteration time to 10ms during iotests - Use separate passphrase for top vs backing file in iotests - Mark 'encryption_key_missing@ field as d

[Qemu-block] [PATCH v5 03/18] qcow: document another weakness of qcow AES encryption

2017-02-21 Thread Daniel P. Berrange
: Alberto Garcia Signed-off-by: Daniel P. Berrange --- qemu-img.texi | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/qemu-img.texi b/qemu-img.texi index 174aae3..db4534b 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -544,16 +544,29 @@ The use of encryption in

[Qemu-block] [PATCH v5 02/18] block: add ability to set a prefix for opt names

2017-02-21 Thread Daniel P. Berrange
at a later date. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- block/crypto.c | 111 + block/crypto.h | 42 +++--- 2 files changed, 119 insertions(+), 34 deletions(-) diff --g

[Qemu-block] [PATCH v5 01/18] block: expose crypto option names / defs to other drivers

2017-02-21 Thread Daniel P. Berrange
: Daniel P. Berrange --- block/crypto.c | 61 +++ block/crypto.h | 91 ++ 2 files changed, 102 insertions(+), 50 deletions(-) create mode 100644 block/crypto.h diff --git a/block/crypto.c b/block/crypto.c

[Qemu-block] [PATCH v5 04/18] qcow: require image size to be > 1 for new images

2017-02-21 Thread Daniel P. Berrange
The qcow driver refuses to open images which are less than 2 bytes in size, but will happily create such images. Add a check in the create path to avoid this discrepancy. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- block

Re: [Qemu-block] [PATCH v4 13/18] qcow2: add support for LUKS encryption format

2017-02-20 Thread Daniel P. Berrange
On Thu, Feb 16, 2017 at 02:42:04PM +0100, Alberto Garcia wrote: > On Fri 10 Feb 2017 06:09:05 PM CET, Daniel P. Berrange wrote: > > @@ -990,12 +1123,6 @@ static int qcow2_open(BlockDriverState *bs, QDict > > *options, int flags, > > s->refcount_max = UINT64_C(1)

[Qemu-block] [PATCH v3 4/4] qemu-img: copy *key-secret opts when opening newly created files

2017-02-20 Thread Daniel P. Berrange
file. Signed-off-by: Daniel P. Berrange --- qemu-img.c | 44 ++-- 1 file changed, 42 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index e48e676..bad19fd 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -317,6 +317,46 @@ static

Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-02-20 Thread Daniel P. Berrange
On Mon, Feb 20, 2017 at 12:46:52PM +, Daniel P. Berrange wrote: > On Fri, Feb 03, 2017 at 11:39:35PM +0100, Max Reitz wrote: > > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > > The qemu-img dd/convert commands will create a image file and > > > then try to o

[Qemu-block] [PATCH v3 1/4] qemu-img: add support for --object with 'dd' command

2017-02-20 Thread Daniel P. Berrange
The qemu-img dd command added --image-opts support, but missed the corresponding --object support. This prevented passing secrets (eg auth passwords) needed by certain disk images. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-img.c | 18 ++ 1 file changed

[Qemu-block] [PATCH v3 3/4] qemu-img: introduce --target-image-opts for 'convert' command

2017-02-20 Thread Daniel P. Berrange
s --target-image-opts to indicate that the target filename includes options. Currently this mandates use of the --skip-create flag too. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 4 +-- qemu-img.c | 84 +++- qemu-img.te

[Qemu-block] [PATCH v3 0/4] Improve convert and dd commands

2017-02-20 Thread Daniel P. Berrange
nd (Max) - Fix error reporting when using compressed image and skip-create (Max) - Unconditionally create QDict when open files (Max) Changed in v2: - Replace dd -n flag with support for conv=nocreat,notrunc - Misc typos (Eric, Fam) Daniel P. Berrange (4): qemu-img: add support for --object

[Qemu-block] [PATCH v3 2/4] qemu-img: fix --image-opts usage with dd command

2017-02-20 Thread Daniel P. Berrange
P. Berrange --- qemu-img.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index 739345e..d8a737f 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -4102,8 +4102,13 @@ static int img_dd(int argc, char **argv) goto out; } -blk2

Re: [Qemu-block] [PATCH v2 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-02-20 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 11:39:35PM +0100, Max Reitz wrote: > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > The qemu-img dd/convert commands will create a image file and > > then try to open it. Historically it has been possible to open > > new files without passing any op

Re: [Qemu-block] [PATCH v2 5/6] qemu-img: introduce --target-image-opts for 'convert' command

2017-02-20 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 11:32:13PM +0100, Max Reitz wrote: > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > The '--image-opts' flags indicates whether the source filename > > includes options. The target filename has to remain in the > > plain filename form

Re: [Qemu-block] [PATCH v2 4/6] qemu-img: add support for -o arg to dd command

2017-02-20 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 11:07:13PM +0100, Max Reitz wrote: > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > The -o arg to the convert command allows specification of format/protocol > > options for the newly created image. This adds a -o arg to the dd command > >

Re: [Qemu-block] [PATCH v2 3/6] qemu-img: add support for conv=nocreat, notrunc args to dd command

2017-02-20 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 10:44:46PM +0100, Max Reitz wrote: > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > The -n arg to the convert command allows use of a pre-existing image, > > rather than creating a new image. This adds equivalent functionality > > to the dd command

Re: [Qemu-block] [PATCH v2 1/6] qemu-img: add support for --object with 'dd' command

2017-02-20 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 10:01:53PM +0100, Max Reitz wrote: > On 03.02.2017 13:02, Daniel P. Berrange wrote: > > The qemu-img dd command added --image-opts support, but missed > > the corresponding --object support. This prevented passing > > secrets (eg auth passwords) n

Re: [Qemu-block] [Qemu-devel] Estimation of qcow2 image size converted from raw image

2017-02-13 Thread Daniel P. Berrange
On Mon, Feb 13, 2017 at 12:03:35PM -0500, John Snow wrote: > Also keep in mind that changing the cluster size will give you different > answers, too -- but that different cluster sizes will effect the runtime > performance of the image as well. This means that apps trying to figure out this future

[Qemu-block] [PATCH v4 17/18] block: remove all encryption handling APIs

2017-02-10 Thread Daniel P. Berrange
whether an image is encrypted or not, since that is a potentially useful piece of metadata to report to the user. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block.c | 77 +-- block/crypto.c| 1 - block

[Qemu-block] [PATCH v4 10/18] qcow2: make qcow2_encrypt_sectors encrypt in place

2017-02-10 Thread Daniel P. Berrange
: Alberto Garcia Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 17 ++--- block/qcow2.c | 4 ++-- block/qcow2.h | 3 +-- 3 files changed, 9 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index 928c1e2..907e869 100644

[Qemu-block] [PATCH v4 09/18] qcow: convert QCow to use QCryptoBlock for encryption

2017-02-10 Thread Daniel P. Berrange
: Daniel P. Berrange --- block/crypto.c | 10 +++ block/crypto.h | 9 +++ block/qcow.c | 183 +++ qapi/block-core.json | 17 - 4 files changed, 116 insertions(+), 103 deletions(-) diff --git a/block/crypto.c b/block/crypto.c

[Qemu-block] [PATCH v4 04/18] qcow: require image size to be > 1 for new images

2017-02-10 Thread Daniel P. Berrange
The qcow driver refuses to open images which are less than 2 bytes in size, but will happily create such images. Add a check in the create path to avoid this discrepancy. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- block

[Qemu-block] [PATCH v4 14/18] qcow2: add iotests to cover LUKS encryption support

2017-02-10 Thread Daniel P. Berrange
This extends the 087 iotest to cover LUKS encryption when doing blockdev-add. Two further tests are added to validate read/write of LUKS encrypted images with a single file and with a backing file. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/087 | 32

[Qemu-block] [PATCH v4 18/18] block: pass option prefix down to crypto layer

2017-02-10 Thread Daniel P. Berrange
to layer can report accurate error messages, we must tell it what option name prefix was used. Signed-off-by: Daniel P. Berrange --- block/crypto.c| 4 ++-- block/qcow.c | 7 --- block/qcow2.c | 15 +-- crypto/block-luks.c | 8 ++

[Qemu-block] [PATCH v4 12/18] qcow2: extend specification to cover LUKS encryption

2017-02-10 Thread Daniel P. Berrange
Update the qcow2 specification to describe how the LUKS header is placed inside a qcow2 file, when using LUKS encryption for the qcow2 payload instead of the legacy AES-CBC encryption Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- docs/specs/qcow2.txt | 95

[Qemu-block] [PATCH v4 16/18] block: rip out all traces of password prompting

2017-02-10 Thread Daniel P. Berrange
Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- h

[Qemu-block] [PATCH v4 13/18] qcow2: add support for LUKS encryption format

2017-02-10 Thread Daniel P. Berrange
sectors when qcow2 internal snapshots are used, thus giving stronger protection against watermarking attacks. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 4 +- block/qcow2-refcount.c | 10 ++ block/qcow2.c | 284 +++

[Qemu-block] [PATCH v4 15/18] iotests: enable tests 134 and 158 to work with qcow (v1)

2017-02-10 Thread Daniel P. Berrange
The 138 and 158 iotests exercise the legacy qcow2 aes encryption code path and they work fine with qcow v1 too. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/158 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff

[Qemu-block] [PATCH v4 08/18] qcow: make encrypt_sectors encrypt in place

2017-02-10 Thread Daniel P. Berrange
Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow.c | 44 +++- 1 file changed, 15 insertions(+), 29 deletions(-) diff --git a/block/qcow.c b/block/qcow.c index 744b25e..f07cdbb 100644 --- a/block/qcow.c +++ b/block/qcow.c @@ -316,11

[Qemu-block] [PATCH v4 11/18] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-02-10 Thread Daniel P. Berrange
CPU state. Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 47 +-- block/qcow2.c | 192 + block/qcow2.h | 5 +- qapi/block-core.json | 8 +- tests/qemu-iotests/049 | 2 +- tests/qemu

[Qemu-block] [PATCH v4 07/18] iotests: fix 097 when run with qcow

2017-02-10 Thread Daniel P. Berrange
ff-by: Daniel P. Berrange --- tests/qemu-iotests/097 | 10 +--- tests/qemu-iotests/097.out | 125 ++-- tests/qemu-iotests/173 | 126 + tests/qemu-iotests/173.out

[Qemu-block] [PATCH v4 03/18] qcow: document another weakness of qcow AES encryption

2017-02-10 Thread Daniel P. Berrange
: Alberto Garcia Signed-off-by: Daniel P. Berrange --- qemu-img.texi | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/qemu-img.texi b/qemu-img.texi index 174aae3..db4534b 100644 --- a/qemu-img.texi +++ b/qemu-img.texi @@ -544,16 +544,29 @@ The use of encryption in

[Qemu-block] [PATCH v4 05/18] iotests: skip 042 with qcow which dosn't support zero sized images

2017-02-10 Thread Daniel P. Berrange
Test 042 is designed to verify operation with zero sized images. Such images are not supported with qcow (v1), so this test has always failed. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/042 | 2 +- 1 file changed, 1 insertion

[Qemu-block] [PATCH v4 06/18] iotests: skip 048 with qcow which doesn't support resize

2017-02-10 Thread Daniel P. Berrange
Test 048 is designed to verify data preservation during an image resize. The qcow (v1) format impl has never supported resize so always fails. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/048 | 2 +- 1 file changed, 1 insertion

[Qemu-block] [PATCH v4 02/18] block: add ability to set a prefix for opt names

2017-02-10 Thread Daniel P. Berrange
at a later date. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange --- block/crypto.c | 111 + block/crypto.h | 42 +++--- 2 files changed, 119 insertions(+), 34 deletions(-) diff --g

[Qemu-block] [PATCH v4 01/18] block: expose crypto option names / defs to other drivers

2017-02-10 Thread Daniel P. Berrange
: Daniel P. Berrange --- block/crypto.c | 61 +++ block/crypto.h | 91 ++ 2 files changed, 102 insertions(+), 50 deletions(-) create mode 100644 block/crypto.h diff --git a/block/crypto.c b/block/crypto.c

[Qemu-block] [PATCH v4 00/18] Convert QCow[2] to QCryptoBlock & add LUKS support

2017-02-10 Thread Daniel P. Berrange
ryption' option - Drop redundant test scenarios - Use small file sizes for iotests - Drop pbkdf iteration time to 10ms during iotests - Use separate passphrase for top vs backing file in iotests - Mark 'encryption_key_missing@ field as deprecated Daniel P. Berrange (18): block: expo

Re: [Qemu-block] [PATCH v3 08/18] qcow: make encrypt_sectors encrypt in place

2017-02-10 Thread Daniel P. Berrange
On Fri, Feb 10, 2017 at 11:44:15AM +0100, Alberto Garcia wrote: > On Thu 26 Jan 2017 11:18:17 AM CET, Daniel P. Berrange wrote: > > Instead of requiring separate input/output buffers for > > encrypting data, change encrypt_sectors() to assume > > use of a single buffer, enc

Re: [Qemu-block] [PATCH v3 0/5] Improve I/O tests coverage of LUKS

2017-02-09 Thread Daniel P. Berrange
ping ? On Tue, Jan 24, 2017 at 11:57:43AM +, Daniel P. Berrange wrote: > The main goal of this series is to get the I/O tests passing > 100% with LUKS when run with './check -luks'. It also adds a > few more combinations to the LUKS/dmcrypt interoperability > test. >

Re: [Qemu-block] [PATCH v3 11/18] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-02-08 Thread Daniel P. Berrange
On Wed, Feb 08, 2017 at 05:15:34PM +0100, Alberto Garcia wrote: > On Thu 26 Jan 2017 11:18:20 AM CET, "Daniel P. Berrange" > wrote: > > > @@ -751,6 +757,23 @@ static int > > qcow2_update_options_prepare(BlockDriverState *bs, > > r->

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-06 Thread Daniel P. Berrange
On Mon, Feb 06, 2017 at 06:24:42PM +0100, Markus Armbruster wrote: > "Daniel P. Berrange" writes: > > > On Mon, Feb 06, 2017 at 04:36:50PM +0100, Markus Armbruster wrote: > >> Kevin Wolf writes: > >> > Want to make use of the shiny new QemuOpts and ge

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-06 Thread Daniel P. Berrange
On Mon, Feb 06, 2017 at 04:36:50PM +0100, Markus Armbruster wrote: > Kevin Wolf writes: > > Want to make use of the shiny new QemuOpts and get things parsed into > > a nested object? Well, provide a real schema instead of "any" then. > > Sadly, this is somewhere between impractical and impossible

Re: [Qemu-block] [libvirt] [Block Replication] Question about supporting COLO in libvirt

2017-02-06 Thread Daniel P. Berrange
On Mon, Feb 06, 2017 at 08:34:28PM +0800, Hailiang Zhang wrote: > Hi, > I'm trying to implement supporting COLO in libvirt, > But i found an annoying problem that libvirt does not > support the command line option argument syntax we used > for block replication in QEMU. > > That is libvirt does no

Re: [Qemu-block] [Qemu-devel] Non-flat command line option argument syntax

2017-02-06 Thread Daniel P. Berrange
On Sat, Feb 04, 2017 at 08:21:50PM +0800, Fam Zheng wrote: > On Thu, 02/02 20:42, Markus Armbruster wrote: > > === Comparison === > > > > In my opinion, dotted keys are weird and ugly, but at least they don't > > add to the quoting mess. Structured values look better, except when > > they do add

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-02-06 Thread Daniel P. Berrange
On Fri, Feb 03, 2017 at 07:56:11PM +0100, Max Reitz wrote: > > In case you say that's inconvenient: pretty much everything about dd's > > archaic user interface is inconvenient. If you want convenient, roll > > your own. If you want familiar, stick to the original. > > I agree. But qemu-img dd a

[Qemu-block] [PATCH v2 5/6] qemu-img: introduce --target-image-opts for 'convert' command

2017-02-03 Thread Daniel P. Berrange
s --target-image-opts to indicate that the target filename includes options. Currently this mandates use of the --skip-create flag too. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 6 +-- qemu-img.c | 131 --- qemu-img.tex

[Qemu-block] [PATCH v2 1/6] qemu-img: add support for --object with 'dd' command

2017-02-03 Thread Daniel P. Berrange
The qemu-img dd command added --image-opts support, but missed the corresponding --object support. This prevented passing secrets (eg auth passwords) needed by certain disk images. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-img.c | 16 1 file changed

[Qemu-block] [PATCH v2 4/6] qemu-img: add support for -o arg to dd command

2017-02-03 Thread Daniel P. Berrange
The -o arg to the convert command allows specification of format/protocol options for the newly created image. This adds a -o arg to the dd command to get feature parity. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 2 +- qemu-img.c | 32 +++- qemu

[Qemu-block] [PATCH v2 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-02-03 Thread Daniel P. Berrange
file. Signed-off-by: Daniel P. Berrange --- qemu-img.c | 51 +++ 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index dc4c6eb..98522dd 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -319,6 +319,49 @@ static

[Qemu-block] [PATCH v2 0/6] qemu-img: improve convert & dd commands

2017-02-03 Thread Daniel P. Berrange
e the 'dd' command to address feature gaps wrt the 'convert' command. The last two patches implement the improvements described above. Changed in v2: - Replace dd -n flag with support for conv=nocreat,notrunc - Misc typos (Eric, Fam) Daniel P. Berrange (6): qemu-img: add sup

[Qemu-block] [PATCH v2 3/6] qemu-img: add support for conv=nocreat, notrunc args to dd command

2017-02-03 Thread Daniel P. Berrange
e truncated to match the required output size. 'conv=notrunc' cna be used to preserve the existing image size. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 4 +- qemu-img.c | 137 +-- qemu-img.texi| 10 +++- 3

[Qemu-block] [PATCH v2 2/6] qemu-img: fix --image-opts usage with dd command

2017-02-03 Thread Daniel P. Berrange
The --image-opts flag can only be used to affect the parsing of the source image. The target image has to be specified in the traditional style regardless, since it needs to be passed to the bdrv_create() API which does not support the new style opts. Signed-off-by: Daniel P. Berrange --- qemu

Re: [Qemu-block] Non-flat command line option argument syntax

2017-02-03 Thread Daniel P. Berrange
On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote: > === Dotted keys === > > One sufficiently powerful syntax extension already exists: the dotted > key convention. It's syntactically unambiguous only when none of the > KEYs involved contains '.' To adopt it across the board, we'

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-02-01 Thread Daniel P. Berrange
On Wed, Feb 01, 2017 at 01:31:01PM +0100, Max Reitz wrote: > On 01.02.2017 13:28, Daniel P. Berrange wrote: > > On Wed, Feb 01, 2017 at 01:23:54PM +0100, Max Reitz wrote: > >> On 01.02.2017 13:16, Daniel P. Berrange wrote: > >>> On Wed, Feb 01, 2017 at 01:13:39PM +01

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-02-01 Thread Daniel P. Berrange
On Wed, Feb 01, 2017 at 01:23:54PM +0100, Max Reitz wrote: > On 01.02.2017 13:16, Daniel P. Berrange wrote: > > On Wed, Feb 01, 2017 at 01:13:39PM +0100, Max Reitz wrote: > >> On 30.01.2017 19:37, Eric Blake wrote: > >>> On 01/26/2017 07:27 AM, Daniel P. Berrange wrot

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-02-01 Thread Daniel P. Berrange
On Wed, Feb 01, 2017 at 01:13:39PM +0100, Max Reitz wrote: > On 30.01.2017 19:37, Eric Blake wrote: > > On 01/26/2017 07:27 AM, Daniel P. Berrange wrote: > >> On Thu, Jan 26, 2017 at 08:35:30PM +0800, Fam Zheng wrote: > >>> On Thu, 01/26 11:04, Daniel P. Berrange w

Re: [Qemu-block] [Qemu-devel] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-01-26 Thread Daniel P. Berrange
On Thu, Jan 26, 2017 at 08:35:30PM +0800, Fam Zheng wrote: > On Thu, 01/26 11:04, Daniel P. Berrange wrote: > > The -n arg to the convert command allows use of a pre-existing image, > > rather than creating a new image. This adds a -n arg to the dd command > > to get f

[Qemu-block] [PATCH v1 5/6] qemu-img: introduce --target-image-opts for 'convert' command

2017-01-26 Thread Daniel P. Berrange
s --target-image-opts to indicate that the target filename includes options. Currently this mandates use of the --skip-create flag too. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 8 ++-- qemu-img.c | 131 --- qemu-img.tex

[Qemu-block] [PATCH v1 4/6] qemu-img: add support for -o arg to dd command

2017-01-26 Thread Daniel P. Berrange
The -o arg to the convert command allows specification of format/protocol options for the newly created image. This adds a -o arg to the dd command to get feature parity. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 4 ++-- qemu-img.c | 32

[Qemu-block] [PATCH v1 6/6] qemu-img: copy *key-secret opts when opening newly created files

2017-01-26 Thread Daniel P. Berrange
created file. Signed-off-by: Daniel P. Berrange --- qemu-img.c | 51 +++ 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index a751781..130cec7 100644 --- a/qemu-img.c +++ b/qemu-img.c @@ -317,6 +317,49 @@ static

[Qemu-block] [PATCH v1 3/6] qemu-img: add support for -n arg to dd command

2017-01-26 Thread Daniel P. Berrange
The -n arg to the convert command allows use of a pre-existing image, rather than creating a new image. This adds a -n arg to the dd command to get feature parity. Signed-off-by: Daniel P. Berrange --- qemu-img-cmds.hx | 4 +-- qemu-img.c | 79

[Qemu-block] [PATCH v1 0/6] qemu-img: improve convert & dd commands

2017-01-26 Thread Daniel P. Berrange
mmand. The last two patches implement the improvements described above. Daniel P. Berrange (6): qemu-img: add support for --object with 'dd' command qemu-img: fix --image-opts usage with dd command qemu-img: add support for -n arg to dd command qemu-img: add support for -o arg to dd co

[Qemu-block] [PATCH v1 1/6] qemu-img: add support for --object with 'dd' command

2017-01-26 Thread Daniel P. Berrange
The qemu-img dd command added --image-opts support, but missed the corresponding --object support. This prevented passing secrets (eg auth passwords) needed by certain disk images. Signed-off-by: Daniel P. Berrange --- qemu-img.c | 16 1 file changed, 16 insertions(+) diff

[Qemu-block] [PATCH v1 2/6] qemu-img: fix --image-opts usage with dd command

2017-01-26 Thread Daniel P. Berrange
The --image-opts flag can only be used to affect the parsing of the source image. The target image has to be specified in the traditional style regardless, since it needs to be passed to the brdv_create() API which does not support the new style opts. Signed-off-by: Daniel P. Berrange --- qemu

[Qemu-block] [PATCH v3 16/18] block: rip out all traces of password prompting

2017-01-26 Thread Daniel P. Berrange
Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- h

[Qemu-block] [PATCH v3 15/18] iotests: enable tests 134 and 158 to work with qcow (v1)

2017-01-26 Thread Daniel P. Berrange
The 138 and 158 iotests exercise the legacy qcow2 aes encryption code path and they work fine with qcow v1 too. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/134 | 2 +- tests/qemu-iotests/158 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff

[Qemu-block] [PATCH v3 17/18] block: remove all encryption handling APIs

2017-01-26 Thread Daniel P. Berrange
whether an image is encrypted or not, since that is a potentially useful piece of metadata to report to the user. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block.c | 77 +-- block/crypto.c| 1 - block

[Qemu-block] [PATCH v3 11/18] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-01-26 Thread Daniel P. Berrange
CPU state. Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 47 +-- block/qcow2.c | 190 + block/qcow2.h | 5 +- qapi/block-core.json | 8 +- tests/qemu-iotests/049 | 2 +- tests/qemu

[Qemu-block] [PATCH v3 14/18] qcow2: add iotests to cover LUKS encryption support

2017-01-26 Thread Daniel P. Berrange
This extends the 087 iotest to cover LUKS encryption when doing blockdev-add. Two further tests are added to validate read/write of LUKS encrypted images with a single file and with a backing file. Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/087 | 32 - tests

[Qemu-block] [PATCH v3 13/18] qcow2: add support for LUKS encryption format

2017-01-26 Thread Daniel P. Berrange
sectors when qcow2 internal snapshots are used, thus giving stronger protection against watermarking attacks. Signed-off-by: Daniel P. Berrange --- block/qcow2-cluster.c | 4 +- block/qcow2-refcount.c | 10 ++ block/qcow2.c | 284 +

[Qemu-block] [PATCH v3 18/18] block: pass option prefix down to crypto layer

2017-01-26 Thread Daniel P. Berrange
to layer can report accurate error messages, we must tell it what option name prefix was used. Signed-off-by: Daniel P. Berrange --- block/crypto.c | 4 ++-- block/qcow.c | 7 --- block/qcow2.c | 15 +-- crypto/block-luks.c| 8 ++-- crypto/blo

[Qemu-block] [PATCH v3 07/18] iotests: fix 097 when run with qcow

2017-01-26 Thread Daniel P. Berrange
ff-by: Daniel P. Berrange --- tests/qemu-iotests/097 | 10 +--- tests/qemu-iotests/097.out | 125 ++-- tests/qemu-iotests/173 | 126 + tests/qemu-iotests/173.out

[Qemu-block] [PATCH v3 06/18] iotests: skip 048 with qcow which doesn't support resize

2017-01-26 Thread Daniel P. Berrange
Test 048 is designed to verify data preservation during an image resize. The qcow (v1) format impl has never supported resize so always fails. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/048 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a

[Qemu-block] [PATCH v3 04/18] qcow: require image size to be > 1 for new images

2017-01-26 Thread Daniel P. Berrange
The qcow driver refuses to open images which are less than 2 bytes in size, but will happily create such images. Add a check in the create path to avoid this discrepancy. Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- block/qcow.c | 6 ++ 1 file changed, 6 insertions(+) diff

<    1   2   3   4   5   6   7   8   9   10   >