Re: [Qemu-block] [PATCH v3 3/3] qemu-iotests: Add test for bz #1745922

2019-09-13 Thread Maxim Levitsky
On Fri, 2019-09-13 at 13:01 +0200, Max Reitz wrote: > On 13.09.19 00:37, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > tests/qemu-iotests/263 | 91 ++ > > tests/qemu-iotests/263.out | 40 + &g

Re: [Qemu-block] [PATCH v2 6/7] curl: Handle success in multi_check_completion

2019-09-13 Thread Maxim Levitsky
On Fri, 2019-09-13 at 13:20 +0200, Max Reitz wrote: > On 10.09.19 18:13, Maxim Levitsky wrote: > > On Tue, 2019-09-10 at 14:41 +0200, Max Reitz wrote: > > > Background: As of cURL 7.59.0, it verifies that several functions are > > > not called from within a callb

[Qemu-block] [PATCH v3 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335

2019-09-12 Thread Maxim Levitsky
: https://bugzilla.redhat.com/show_bug.cgi?id=1745922 CC: qemu-stable V2: grammar, spelling and code style fixes. V3: more fixes after the review. Best regards, Maxim Levitsky Maxim Levitsky (3): block/qcow2: refactoring of threaded encryption code block/qcow2: fix the corruption when

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] LUKS: better error message when creating too large files

2019-09-12 Thread Maxim Levitsky
On Mon, 2019-07-22 at 10:05 +0100, Daniel P. Berrangé wrote: > On Sun, Jul 21, 2019 at 09:15:07PM +0300, Maxim Levitsky wrote: > > Currently if you attampt to create too large file with luks you > > get the following error message: > > > > Formatting 'test.luks', fmt=l

[Qemu-block] [PATCH v3 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-12 Thread Maxim Levitsky
This commit tries to clarify few function arguments, and add comments describing the encrypt/decrypt interface Signed-off-by: Maxim Levitsky --- block/qcow2-cluster.c | 8 +++--- block/qcow2-threads.c | 63 ++- 2 files changed, 54 insertions(+), 17

[Qemu-block] [PATCH v2 09/11] block/qcow2: implement blockdev-amend

2019-09-12 Thread Maxim Levitsky
Currently only for changing crypto parameters Signed-off-by: Maxim Levitsky --- block/qcow2.c| 71 qapi/block-core.json | 6 ++-- 2 files changed, 75 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 26f83aeb44

[Qemu-block] [PATCH v3 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

2019-09-12 Thread Maxim Levitsky
, and causes some files with zero areas to contain garbage there instead. But as described above it can happen elsewhere as well Signed-off-by: Maxim Levitsky Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-cluster.c | 29 + 1 file changed, 17 insertions

[Qemu-block] [PATCH v2 08/11] block/crypto: implement blockdev-amend

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 85 ++-- qapi/block-core.json | 7 ++-- 2 files changed, 71 insertions(+), 21 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index f42fa057e6..5905f7f520

[Qemu-block] [PATCH v2 07/11] block: add x-blockdev-amend qmp command

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/Makefile.objs | 2 +- block/amend.c | 116 ++ include/block/block_int.h | 23 ++-- qapi/block-core.json | 26 + qapi/job.json | 4 +- 5 files changed, 163 insertions

[Qemu-block] [PATCH v3 3/3] qemu-iotests: Add test for bz #1745922

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/263 | 91 ++ tests/qemu-iotests/263.out | 40 + tests/qemu-iotests/group | 1 + 3 files changed, 132 insertions(+) create mode 100755 tests/qemu-iotests/263 create mode 100644 tests

[Qemu-block] [PATCH v2 05/11] block/crypto: implement the encryption key management

2019-09-12 Thread Maxim Levitsky
, even readonly, this will fail. Also thanks to Daniel Berrange for the variant of that hack that involves asking for read, rather that write permission Signed-off-by: Maxim Levitsky --- block/crypto.c | 118 +++-- 1 file changed, 115 insertions(+), 3

[Qemu-block] [PATCH v2 10/11] iotests: filter few more luks specific create options

2019-09-12 Thread Maxim Levitsky
Those options are test input anyway, and this allows more tests to be able to have same output on both qcow2 luks encrypted images and raw luks images Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/087.out | 6 +++--- tests/qemu-iotests/134.out | 2 +- tests/qemu-iotests/158

[Qemu-block] [PATCH v2 11/11] iotests : add tests for encryption key management

2019-09-12 Thread Maxim Levitsky
Note that currently I add tests 300-302, which are placeholders to ease the rebase. In final version of these patches I will update these. Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/300 | 202 + tests/qemu-iotests/300.out | 98 +++ tests

[Qemu-block] [PATCH v2 02/11] qcrypto-luks: extend the create options for upcoming encryption key management

2019-09-12 Thread Maxim Levitsky
Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it. Check that active=true it when creating. Signed-off-by: Maxim Levitsky --- block/crypto.c | 2 ++ block/crypto.h | 16

[Qemu-block] [PATCH v2 06/11] qcow2: implement crypto amend options

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/qcow2.c | 77 +-- 1 file changed, 62 insertions(+), 15 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 0618a63793..26f83aeb44 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -172,6 +172,25

[Qemu-block] [PATCH v2 00/11] RFC crypto/luks: encryption key managment using amend interface

2019-09-12 Thread Maxim Levitsky
device, while preveting the key managment from happening in this case, as it is unsafe. I added a new iotest dedicated to that as well. Best regards, Maxim Levitsky Maxim Levitsky (11): qcrypto: add suport for amend options qcrypto-luks: extend the create options for upcoming

[Qemu-block] [PATCH v2 04/11] block: amend: add 'force' option

2019-09-12 Thread Maxim Levitsky
'force' optinion will be used for some unsafe option amend operations. This includes things like erasing last keyslot in luks (which pretty much guarantees destroying the data, unless the master key is backed up by extrnal means, but that _might_ be desired result) Signed-off-by: Maxim

[Qemu-block] [PATCH v2 01/11] qcrypto: add suport for amend options

2019-09-12 Thread Maxim Levitsky
This adds the qcrypto_amend_options and corresponding crypto driver callbacks for the for encrypted key managedment Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block.c | 31 +++ crypto/blockpriv.h | 8 include

[Qemu-block] [PATCH v2 03/11] qcrypto-luks: implement the encryption key management

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 356 +++- 1 file changed, 354 insertions(+), 2 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index fed80e6646..26ce50b111 100644 --- a/crypto/block-luks.c +++ b/crypto/block

Re: [Qemu-block] [Qemu-devel] [PATCH 09/10] block/qcow2: implement blockdev-amend

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 15:12 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:07PM +0300, Maxim Levitsky wrote: > > Currently only for changing crypto parameters > > > > Signed-off-by: Maxim Levitsky > > --- >

Re: [Qemu-block] [Qemu-devel] [PATCH 08/10] block/crypto: implement blockdev-amend

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 15:10 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:06PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > block/crypto.c | 86 +--- > > qapi/block-core

Re: [Qemu-block] [Qemu-devel] [PATCH 06/10] qcow2: implement crypto amend options

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 15:06 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:04PM +0300, Maxim Levitsky wrote: > > --- > > block/qcow2.c | 79 --- > > 1 file changed, 63 insertions(+), 16 deletions(-) > &g

Re: [Qemu-block] [Qemu-devel] [PATCH 05/10] block/crypto: implement the encryption key management

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 15:04 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:03PM +0300, Maxim Levitsky wrote: > > This implements the encryption key management > > using the generic code in qcrypto layer > > (currently only for qemu-img amend) > >

Re: [Qemu-block] [Qemu-devel] [PATCH 04/10] block: amend: add 'force' option

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:59 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:02PM +0300, Maxim Levitsky wrote: > > This could do with some text to explain what this will be > used for. I actually added an explanation to the man page " +--force allows some

Re: [Qemu-block] [Qemu-devel] [PATCH 03/10] qcrypto-luks: implement the encryption key management

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:55 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:01PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > crypto/block-luks.c | 366 +++- > > 1 file changed, 364

[Qemu-block] [PATCH 11/12] qcrypto-luks: simplify the math used for keyslot locations

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.c | 63 - 1 file changed, 40 insertions(+), 23 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index c6045da33e..0d155c6614 100644 --- a/crypto

[Qemu-block] [PATCH 07/12] qcrypto-luks: purge unused error codes from open callback

2019-09-12 Thread Maxim Levitsky
These values are not used by generic crypto code anyway Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 26 +- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index f3bfc921b2..ba63e9b442 100644

[Qemu-block] [PATCH 10/12] qcrypto-luks: extract store key function

2019-09-12 Thread Maxim Levitsky
This function will be used later to store new keys to the luks metadata Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 304 ++-- 1 file changed, 181 insertions(+), 123 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index

[Qemu-block] [PATCH 12/12] qcrypto-luks: more rigorous header checking

2019-09-12 Thread Maxim Levitsky
Check that keyslots don't overlap with the data, and check that keyslots don't overlap with each other. (this is done using naive O(n^2) nested loops, but since there are just 8 keyslots, this doesn't really matter. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 52

[Qemu-block] [PATCH 03/12] qcrypto-luks: don't overwrite cipher_mode in header

2019-09-12 Thread Maxim Levitsky
This way we can store the header we loaded, which will be used in key management code Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c

[Qemu-block] [PATCH 09/12] qcrypto-luks: extract check and parse header

2019-09-12 Thread Maxim Levitsky
This is just to make qcrypto_block_luks_open more reasonable in size. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 235 1 file changed, 127 insertions(+), 108 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index

[Qemu-block] [PATCH 06/12] qcrypto-luks: use the parsed encryption settings in QCryptoBlockLUKS

2019-09-12 Thread Maxim Levitsky
Prior to that patch, the parsed encryption settings were already stored into the QCryptoBlockLUKS but not used anywhere but in qcrypto_block_luks_get_info Using them simplifies the code Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.c | 169

[Qemu-block] [PATCH 01/12] block-crypto: misc refactoring

2019-09-12 Thread Maxim Levitsky
* rename the write_func to create_write_func, and init_func to create_init_func this is preparation for other write_func that will be used to update the encryption keys. No functional changes Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 12

[Qemu-block] [PATCH 08/12] qcrypto-luks: extract store and load header

2019-09-12 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 158 ++-- 1 file changed, 94 insertions(+), 64 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index ba63e9b442..c3f3488222 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH 05/12] qcrypto-luks: pass keyslot index rather that pointer to the keyslot

2019-09-12 Thread Maxim Levitsky
Another minor refactoring Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 9e59a791a6..b759cc8d19 100644 --- a/crypto/block-luks.c

[Qemu-block] [PATCH 02/12] qcrypto-luks: rename some fields in QCryptoBlockLUKSHeader

2019-09-12 Thread Maxim Levitsky
* key_bytes -> master_key_len * payload_offset = payload_offset_sector (to emphasise that this isn't byte offset) * key_offset -> key_offset_sector - same as above for luks slots Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.

[Qemu-block] [PATCH 00/12] crypto/luks: preparation for encryption key managment

2019-09-12 Thread Maxim Levitsky
since these values are not used anyway. Best regards, Maxim Levitsky Maxim Levitsky (12): block-crypto: misc refactoring qcrypto-luks: rename some fields in QCryptoBlockLUKSHeader qcrypto-luks: don't overwrite cipher_mode in header qcrypto-luks: simplify masterkey and masterkey

[Qemu-block] [PATCH 04/12] qcrypto-luks: simplify masterkey and masterkey length

2019-09-12 Thread Maxim Levitsky
Let the caller allocate masterkey Always use master key len from the header Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- crypto/block-luks.c | 44 +--- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/crypto/block

Re: [Qemu-block] [Qemu-devel] [PATCH v2 13/13] qcrypto-luks: implement more rigorous header checking

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:34 +0100, Daniel P. Berrangé wrote: > On Mon, Aug 26, 2019 at 04:51:03PM +0300, Maxim Levitsky wrote: > > Check that keyslots don't overlap with the data, > > and check that keyslots don't overlap with each other. > > (this is done using naiv

Re: [Qemu-block] [Qemu-devel] [PATCH v2 11/13] qcrypto-luks: refactoring: simplify the math used for keyslot locations

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:17 +0100, Daniel P. Berrangé wrote: > On Mon, Aug 26, 2019 at 04:51:01PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > crypto/block-luks.c | 64 + > > 1 file changed, 41

Re: [Qemu-block] [Qemu-devel] [PATCH v2 09/13] qcrypto-block: extract check and parse header

2019-09-12 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:11 +0100, Daniel P. Berrangé wrote: > On Mon, Aug 26, 2019 at 04:50:59PM +0300, Maxim Levitsky wrote: > > This is just to make qcrypto_block_luks_open more > > reasonable in size. > > > > Signed-off-by: Maxim Levitsky > > -

Re: [Qemu-block] [PATCH v2 5/7] curl: Report only ready sockets

2019-09-10 Thread Maxim Levitsky
{ > -r = curl_multi_socket_action(s->s->multi, socket->fd, 0, > ); > -} while (r == CURLM_CALL_MULTI_PERFORM); > -} > +do { > +r = curl_multi_socket_action(s->multi, socket->fd, 0, ); > +} while (r == CURLM_CALL_MULTI_PERFORM); > } > > static void curl_multi_do(void *arg) Other than that nitpick, Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 7/7] curl: Check curl_multi_add_handle()'s return code

2019-09-10 Thread Maxim Levitsky
uld myself make this patch #1 in the series, since it doesn't depend on others and it itself a bugfix. But this is my style, so I don't mind if you leave this as is. Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 4/7] curl: Pass CURLSocket to curl_multi_do()

2019-09-10 Thread Maxim Levitsky
DRVCURLState *s = socket->state->s; > > -qemu_mutex_lock(>s->mutex); > -curl_multi_do_locked(s); > -curl_multi_check_completion(s->s); > -qemu_mutex_unlock(>s->mutex); > + qemu_mutex_lock(>mutex); > + curl_multi_do_locked(socket); > +curl_multi_check_completion(s); > +qemu_mutex_unlock(>mutex); > } > > static void curl_multi_timeout_do(void *arg) Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 6/7] curl: Handle success in multi_check_completion

2019-09-10 Thread Maxim Levitsky
emu_iovec_memset(acb->qiov, offset, 0, > + acb->bytes - offset); > +} Original code was memsetting the tail of the buffer before waking up the coroutine. Is this change intended? aio_co_wake doesn't enter the co-routine if already in coroutine, but I think that this is an aio fd handler with isn't run in co-routine itself, so the callback could run with not yet ready data. > } > + > +acb->ret = error ? -EIO : 0; > +state->acb[i] = NULL; > +qemu_mutex_unlock(>mutex); > +aio_co_wake(acb->co); > +qemu_mutex_lock(>mutex); > } > > curl_clean_state(state); Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 3/7] curl: Check completion in curl_multi_do()

2019-09-10 Thread Maxim Levitsky
ry, so I probably missed something important. Other than that, Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky > > Signed-off-by: Max Reitz > --- > block/curl.c | 14 ++ > 1 file changed, 2 insertions(+), 12 deletions(-) > > diff --git a/b

Re: [Qemu-block] [PATCH v2 2/7] curl: Keep *socket until the end of curl_sock_cb()

2019-09-10 Thread Maxim Levitsky
_curl_sock_cb(action, (int)fd); > switch (action) { > @@ -207,6 +202,11 @@ static int curl_sock_cb(CURL *curl, curl_socket_t fd, > int action, > break; > } > > +if (action == CURL_POLL_REMOVE) { > +QLIST_REMOVE(socket, next); > +g_free(socket); > +} > + > return 0; > } > Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 1/7] curl: Keep pointer to the CURLState in CURLSocket

2019-09-10 Thread Maxim Levitsky
> socket->fd = fd; > +socket->state = state; > QLIST_INSERT_HEAD(>sockets, socket, next); > } > socket = NULL; Reviewed-by: Maxim Levitsky Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

2019-09-10 Thread Maxim Levitsky
rently an active maintainer. > > Looks good to me with the changes you pointed out (especially res30; > leaving out the unused macros is not so important). All right, I'll send an updated version of those two patches soon. Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-10 Thread Maxim Levitsky
On Tue, 2019-09-10 at 14:17 +, Vladimir Sementsov-Ogievskiy wrote: > 10.09.2019 15:31, Maxim Levitsky wrote: > > On Sat, 2019-09-07 at 19:08 +, Vladimir Sementsov-Ogievskiy wrote: > > > 06.09.2019 22:57, Maxim Levitsky wrote: > > > > This commit tries to

Re: [Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-10 Thread Maxim Levitsky
On Sat, 2019-09-07 at 19:08 +, Vladimir Sementsov-Ogievskiy wrote: > 06.09.2019 22:57, Maxim Levitsky wrote: > > This commit tries to clarify few function arguments, > > and add comments describing the encrypt/decrypt interface > > > > Signed-off-by: Maxim Levitsky

Re: [Qemu-block] [Qemu-stable] [Qemu-devel] [PATCH 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

2019-09-10 Thread Maxim Levitsky
lucky, you can still tell in gdb at least if the bug is > reproducible, but I wouldn't be surprised if in release builds, half of > the variables were actually optimised away, so that even this wouldn't > work. Agreed. I guess I'll keep the separate asserts anyway after all, even though I prefer shorter code. Best regards, Maxim Levitsky

Re: [Qemu-block] [PATCH v2 3/3] qemu-iotests: Add test for bz #1745922

2019-09-10 Thread Maxim Levitsky
On Mon, 2019-09-09 at 11:35 +0100, Daniel P. Berrangé wrote: > On Fri, Sep 06, 2019 at 10:57:50PM +0300, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > tests/qemu-iotests/263 | 75 ++ > > test

Re: [Qemu-block] [PATCH 0/7] block: Generic file creation fallback

2019-09-10 Thread Maxim Levitsky
On Tue, 2019-09-10 at 11:16 +0200, Max Reitz wrote: > On 05.09.19 15:30, Maxim Levitsky wrote: > > On Fri, 2019-07-12 at 19:35 +0200, Max Reitz wrote: > > > Hi, > > > > > > Kevin commented on my RFC, so I got what an RFC wants, and he didn’t > > >

Re: [Qemu-block] [Qemu-devel] IOTEST 162

2019-09-10 Thread Maxim Levitsky
On Mon, 2019-09-09 at 13:24 -0400, John Snow wrote: > > On 9/6/19 1:25 PM, Maxim Levitsky wrote: > > Hi! > > > > I just had a very fun rabbit hole dive, and I want to share it with you. > > > > I notice for some time that iotest 162 fails with that: >

[Qemu-block] [PATCH v2 3/3] qemu-iotests: Add test for bz #1745922

2019-09-06 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/263 | 75 ++ tests/qemu-iotests/263.out | 19 ++ tests/qemu-iotests/group | 1 + 3 files changed, 95 insertions(+) create mode 100755 tests/qemu-iotests/263 create mode 100644 tests/qemu

[Qemu-block] [PATCH v2 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

2019-09-06 Thread Maxim Levitsky
, and causes some files with zero areas to contain garbage there instead. But as described above it can happen elsewhere as well Signed-off-by: Maxim Levitsky --- block/qcow2-cluster.c | 28 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/block/qcow2

[Qemu-block] [PATCH v2 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-06 Thread Maxim Levitsky
This commit tries to clarify few function arguments, and add comments describing the encrypt/decrypt interface Signed-off-by: Maxim Levitsky --- block/qcow2-cluster.c | 10 +++ block/qcow2-threads.c | 61 ++- 2 files changed, 53 insertions(+), 18

[Qemu-block] [PATCH v2 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335

2019-09-06 Thread Maxim Levitsky
: https://bugzilla.redhat.com/show_bug.cgi?id=1745922 CC: qemu-stable V2: grammar, spelling and code style fixes. Best regards, Maxim Levitsky Maxim Levitsky (3): block/qcow2: refactoring of threaded encryption code block/qcow2: fix the corruption when rebasing luks encrypted files qemu

Re: [Qemu-block] [Qemu-devel] [PATCH 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:17 -0500, Eric Blake wrote: > On 9/6/19 12:32 PM, Maxim Levitsky wrote: > > This fixes subltle corruption introduced by luks threaded encryption > > subtle I usually put the commit messages to a spellchecker, but this time I forgot to do t

Re: [Qemu-block] [Qemu-devel] [PATCH 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:00 -0500, Eric Blake wrote: > On 9/6/19 1:55 PM, Maxim Levitsky wrote: > > > > > +/* > > > > + * qcow2_co_encrypt() > > > > + * > > > > + * Encrypts a sector size aligned contiguous area > > > > + * > &g

Re: [Qemu-block] [Qemu-devel] [PATCH 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 13:00 -0500, Eric Blake wrote: > On 9/6/19 12:31 PM, Maxim Levitsky wrote: > > This commit tries to clarify few function arguments, > > and add comments describing the encrypt/decrypt interface > > > > Signed-off-by: Maxim Levitsky > > --

Re: [Qemu-block] [Qemu-devel] [PATCH 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 20:31 +0300, Maxim Levitsky wrote: > Commit 8ac0f15f335 accidently broke the COW of non changed areas > of newly allocated clusters, when the write spans multiple clusters, > and needs COW both prior and after the write. > This results in 'after' COW area beei

[Qemu-block] [PATCH 2/3] block/qcow2: fix the corruption when rebasing luks encrypted files

2019-09-06 Thread Maxim Levitsky
offset from the host_offset And thus pass correctly to the qcow2_co_encrypt, the host cluster offset and full guest offset Signed-off-by: Maxim Levitsky --- block/qcow2-cluster.c | 26 +++--- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/block/qcow2

[Qemu-block] [PATCH 3/3] qemu-iotests: test for bz #1745922

2019-09-06 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/263 | 76 ++ tests/qemu-iotests/263.out | 19 ++ tests/qemu-iotests/group | 1 + 3 files changed, 96 insertions(+) create mode 100755 tests/qemu-iotests/263 create mode 100644 tests/qemu

[Qemu-block] [PATCH 1/3] block/qcow2: refactoring of threaded encryption code

2019-09-06 Thread Maxim Levitsky
This commit tries to clarify few function arguments, and add comments describing the encrypt/decrypt interface Signed-off-by: Maxim Levitsky --- block/qcow2-cluster.c | 8 +++ block/qcow2-threads.c | 53 ++- 2 files changed, 46 insertions(+), 15

[Qemu-block] [PATCH 0/3] Fix qcow2+luks corruption introduced by commit 8ac0f15f335

2019-09-06 Thread Maxim Levitsky
: https://bugzilla.redhat.com/show_bug.cgi?id=1745922 CC: qemu-stable Best regards, Maxim Levitsky Maxim Levitsky (3): block/qcow2: refactoring of threaded encryption code block/qcow2: fix the corruption when rebasing luks encrypted files qemu-iotests: test for bz #1745922 block/qcow2

Re: [Qemu-block] [PATCH 10/10] iotests : add tests for encryption key management

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 15:14 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:08PM +0300, Maxim Levitsky wrote: > > Note that currently I add tests 300-302, which are > > placeholders to ease the rebase. In final version > > of these patches I will update these

Re: [Qemu-block] [PATCH 02/10] qcrypto-luks: extend the create options for upcoming encryption key management

2019-09-06 Thread Maxim Levitsky
On Fri, 2019-09-06 at 14:49 +0100, Daniel P. Berrangé wrote: > On Fri, Aug 30, 2019 at 11:56:00PM +0300, Maxim Levitsky wrote: > > Now you can specify which slot to put the encryption key to > > Plus add 'active' option which will let user erase the key secret > > instead of

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

2019-09-05 Thread Maxim Levitsky
On Thu, 2019-09-05 at 13:27 -0400, John Snow wrote: > > On 9/5/19 9:24 AM, Maxim Levitsky wrote: > > On Wed, 2019-08-28 at 12:03 +0300, Maxim Levitsky wrote: > > > On Tue, 2019-08-27 at 18:29 -0400, John Snow wrote: > > > > > > > > On 8/25/19 3:15 A

Re: [Qemu-block] [PATCH 0/7] block: Generic file creation fallback

2019-09-05 Thread Maxim Levitsky
; series.) > > So as in the RFC, this series adds a fallback path for creating files > (on the protocol layer) if the protocol driver does not support file > creation, but the file already exists. > Hi! Do you have any update on this patch series by a chance? Best regards, Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

2019-09-05 Thread Maxim Levitsky
On Wed, 2019-08-28 at 12:03 +0300, Maxim Levitsky wrote: > On Tue, 2019-08-27 at 18:29 -0400, John Snow wrote: > > > > On 8/25/19 3:15 AM, Maxim Levitsky wrote: > > > Signed-off-by: Maxim Levitsky > > > -

[Qemu-block] [PATCH 07/10] block: add x-blockdev-amend qmp command

2019-08-30 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/Makefile.objs | 2 +- block/amend.c | 116 ++ include/block/block_int.h | 23 ++-- qapi/block-core.json | 26 + qapi/job.json | 4 +- 5 files changed, 163 insertions

[Qemu-block] [PATCH 04/10] block: amend: add 'force' option

2019-08-30 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block.c | 4 +++- block/qcow2.c | 1 + include/block/block.h | 1 + include/block/block_int.h | 1 + qemu-img-cmds.hx | 4 ++-- qemu-img.c| 8 +++- qemu-img.texi | 6 +- 7 files

[Qemu-block] [PATCH 10/10] iotests : add tests for encryption key management

2019-08-30 Thread Maxim Levitsky
Note that currently I add tests 300-302, which are placeholders to ease the rebase. In final version of these patches I will update these. Signed-off-by: Maxim Levitsky --- tests/qemu-iotests/087.out | 6 +- tests/qemu-iotests/134.out | 2 +- tests/qemu-iotests/158.out

[Qemu-block] [PATCH 02/10] qcrypto-luks: extend the create options for upcoming encryption key management

2019-08-30 Thread Maxim Levitsky
Now you can specify which slot to put the encryption key to Plus add 'active' option which will let user erase the key secret instead of adding it. Check that it is true for creation Signed-off-by: Maxim Levitsky --- block/crypto.c | 2 ++ block/crypto.h | 16

[Qemu-block] [PATCH 09/10] block/qcow2: implement blockdev-amend

2019-08-30 Thread Maxim Levitsky
Currently only for changing crypto parameters Signed-off-by: Maxim Levitsky --- block/qcow2.c| 71 qapi/block-core.json | 4 +-- 2 files changed, 73 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 8dff4c6b5f

[Qemu-block] [PATCH 08/10] block/crypto: implement blockdev-amend

2019-08-30 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/crypto.c | 86 +--- qapi/block-core.json | 4 +-- 2 files changed, 68 insertions(+), 22 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index dbd95a99ba..9cb668ff0e 100644 --- a/block/crypto.c +++ b

[Qemu-block] [PATCH 06/10] qcow2: implement crypto amend options

2019-08-30 Thread Maxim Levitsky
--- block/qcow2.c | 79 --- 1 file changed, 63 insertions(+), 16 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 376bb416fd..8dff4c6b5f 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -171,6 +171,25 @@ static ssize_t

[Qemu-block] [PATCH 01/10] qcrypto: add suport for amend options

2019-08-30 Thread Maxim Levitsky
This adds the qcrypto_amend_options and corresponding crypto driver callbacks for the for encrypted key managedment Signed-off-by: Maxim Levitsky --- crypto/block.c | 31 +++ crypto/blockpriv.h | 8 include/crypto/block.h | 22

[Qemu-block] [PATCH 03/10] qcrypto-luks: implement the encryption key management

2019-08-30 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 366 +++- 1 file changed, 364 insertions(+), 2 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index ba20d55246..21325fbc79 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH 05/10] block/crypto: implement the encryption key management

2019-08-30 Thread Maxim Levitsky
of the luks driver, but during regular use, we have it, and should use it instead. Signed-off-by: Maxim Levitsky --- block/crypto.c | 106 +++-- 1 file changed, 103 insertions(+), 3 deletions(-) diff --git a/block/crypto.c b/block/crypto.c index a6a3e1f1d8

[Qemu-block] [PATCH 00/10] RFC crypto/luks: encryption key managment using amend interface

2019-08-30 Thread Maxim Levitsky
kill if I did something obviously wrong. I did run the iotests - all luks and qcow2 tests, including 3 that I added. Only test 162 seems pretty much always to fail,regardless of my changes I suspect something nbd related / or an enviroment issue Best regards, Maxim Levitsky Maxim Levitsky

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

2019-08-28 Thread Maxim Levitsky
On Tue, 2019-08-27 at 18:29 -0400, John Snow wrote: > > On 8/25/19 3:15 AM, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > block/nvme.c | 83 ++ > > block/trace-events | 2 ++ >

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block/nvme: add support for write zeros

2019-08-28 Thread Maxim Levitsky
s still a hint though). > > On 8/25/19 3:15 AM, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > block/nvme.c | 72 +++- > > block/trace-events | 1 + > > include/block/nvme.h | 19 +++

Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/13] introduce g_autowipe

2019-08-27 Thread Maxim Levitsky
On Tue, 2019-08-27 at 11:52 +0100, Daniel P. Berrangé wrote: > On Mon, Aug 26, 2019 at 04:50:51PM +0300, Maxim Levitsky wrote: > > Marking a pointer with g_autowipe, will > > not only free it at the scope exit, but also > > erase the data it points to

[Qemu-block] [PATCH v2 12/13] qcrypto-luks: use g_autowipe

2019-08-26 Thread Maxim Levitsky
close. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 61 - 1 file changed, 22 insertions(+), 39 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 6a43d97ce5..db0fb764b4 100644 --- a/crypto/block-luks.c +++ b/crypto

[Qemu-block] [PATCH v2 06/13] qcrypto-block: pass keyslot index rather that pointer to the keyslot

2019-08-26 Thread Maxim Levitsky
Another minor refactoring Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index 331377293d..0d81f2ac61 100644 --- a/crypto/block-luks.c +++ b/crypto/block-luks.c @@ -410,7

[Qemu-block] [PATCH v2 05/13] qcrypto-luks: simplify masterkey and masterkey length

2019-08-26 Thread Maxim Levitsky
Let the caller allocate masterkey Always use master key len from the header Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 44 +--- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index

[Qemu-block] [PATCH v2 09/13] qcrypto-block: extract check and parse header

2019-08-26 Thread Maxim Levitsky
This is just to make qcrypto_block_luks_open more reasonable in size. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 254 +--- 1 file changed, 146 insertions(+), 108 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index

[Qemu-block] [PATCH v2 13/13] qcrypto-luks: implement more rigorous header checking

2019-08-26 Thread Maxim Levitsky
Check that keyslots don't overlap with the data, and check that keyslots don't overlap with each other. (this is done using naive O(n^2) nested loops, but since there are just 8 keyslots, this doesn't really matter. Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 46

[Qemu-block] [PATCH v2 11/13] qcrypto-luks: refactoring: simplify the math used for keyslot locations

2019-08-26 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 64 + 1 file changed, 41 insertions(+), 23 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index d713125925..6a43d97ce5 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH v2 08/13] qcrypto-luks: extract store and load header

2019-08-26 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 166 +++- 1 file changed, 102 insertions(+), 64 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index cad65ae0aa..b4dc6fc899 100644 --- a/crypto/block-luks.c +++ b/crypto/block

[Qemu-block] [PATCH v2 10/13] qcrypto-luks: refactoring: extract store key function

2019-08-26 Thread Maxim Levitsky
This function will be used later to store new keys to the luks metadata Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 310 ++-- 1 file changed, 184 insertions(+), 126 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index

[Qemu-block] [PATCH v2 04/13] qcrypto-luks: don't overwrite cipher_mode in header

2019-08-26 Thread Maxim Levitsky
This way we can store the header we loaded, which will be used in key management code Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 9 ++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/crypto/block-luks.c b/crypto/block-luks.c index f12fa2d270..e9ae3f6baa 100644

[Qemu-block] [PATCH v2 01/13] introduce g_autowipe

2019-08-26 Thread Maxim Levitsky
for the information about the fact that plain memset is usually optimized away. Suggested-by: Daniel P. Berrangé Suggested-by: Nir Soffer Signed-off-by: Maxim Levitsky --- include/autowipe.h | 52 ++ 1 file changed, 52 insertions(+) create mode 100644 include

[Qemu-block] [PATCH v2 00/13] RFC crypto/luks: preparation for encryption key managment

2019-08-26 Thread Maxim Levitsky
was given. Best regards, Maxim Levitsky Maxim Levitsky (13): introduce g_autowipe block-crypto: misc refactoring qcrypto-luks: rename some fields in QCryptoBlockLUKSHeader qcrypto-luks: don't overwrite cipher_mode in header qcrypto-luks: simplify masterkey and masterkey length

[Qemu-block] [PATCH v2 07/13] qcrypto-luks: use the parsed encryption settings in QCryptoBlockLUKS

2019-08-26 Thread Maxim Levitsky
Prior to that patch, the parsed encryptio settings were alrady stored into the QCryptoBlockLUKS but not used anywhere but in qcrypto_block_luks_get_info Using them simplifies the code Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 169 +--- 1

[Qemu-block] [PATCH v2 03/13] qcrypto-luks: rename some fields in QCryptoBlockLUKSHeader

2019-08-26 Thread Maxim Levitsky
* key_bytes -> master_key_len * payload_offset = payload_offset_sector (to emphasise that this isn't byte offset) * key_offset -> key_offset_sector - same as above for luks slots Signed-off-by: Maxim Levitsky --- crypto/block-luks.c | 91 +++--

[Qemu-block] [PATCH v2 02/13] block-crypto: misc refactoring

2019-08-26 Thread Maxim Levitsky
* rename the write_func to create_write_func, and init_func to create_init_func this is preparation for other write_func that will be used to update the encryption keys. No functional changes Signed-off-by: Maxim Levitsky Reviewed-by: Daniel P. Berrangé --- block/crypto.c | 12

<    3   4   5   6   7   8   9   10   >