Re: [Qemu-block] [PATCH] nvme: Add support for Read Data and Write Data in CMBs.

On Tue, Jun 13, 2017 at 04:08:35AM -0600, sba...@raithlin.com wrote: > From: Stephen Bates > > Add the ability for the NVMe model to support both the RDS and WDS > modes in the Controller Memory Buffer. > > Although not currently supported in the upstreamed Linux kernel a

[Qemu-block] [PATCH v9 17/20] block: remove all encryption handling APIs

Now that all encryption keys must be provided upfront via the QCryptoSecret API and associated block driver properties there is no need for any explicit encryption handling APIs in the block layer. Encryption can be handled transparently within the block driver. We only retain an API for querying

[Qemu-block] [PATCH v9 19/20] qcow2: report encryption specific image information

Currently 'qemu-img info' reports a simple "encrypted: yes" field. This is not very useful now that qcow2 can support multiple encryption formats. Users want to know which format is in use and some data related to it. Wire up usage of the qcrypto_block_get_info() method so that 'qemu-img info'

[Qemu-block] [PATCH v9 16/20] block: rip out all traces of password prompting

Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz

[Qemu-block] [PATCH v9 15/20] iotests: enable tests 134 and 158 to work with qcow (v1)

The 138 and 158 iotests exercise the legacy qcow2 aes encryption code path and they work fine with qcow v1 too. Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz Signed-off-by: Daniel P. Berrange --- tests/qemu-iotests/134 | 2 +-

[Qemu-block] [PATCH v9 11/20] qcow2: convert QCow2 to use QCryptoBlock for encryption

This converts the qcow2 driver to make use of the QCryptoBlock APIs for encrypting image content, using the legacy QCow2 AES scheme. With this change it is now required to use the QCryptoSecret object for providing passwords, instead of the current block password APIs / interactive prompting.

[Qemu-block] [PATCH v9 18/20] block: pass option prefix down to crypto layer

While the crypto layer uses a fixed option name "key-secret", the upper block layer may have a prefix on the options. e.g. "encrypt.key-secret", in order to avoid clashes between crypto option names & other block option names. To ensure the crypto layer can report accurate error messages, we must

[Qemu-block] [PATCH v9 20/20] docs: document encryption options for qcow, qcow2 and luks

Expand the image format docs to cover the new options for the qcow, qcow2 and luks disk image formats Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-doc.texi | 123

[Qemu-block] [PATCH v9 14/20] qcow2: add iotests to cover LUKS encryption support

This extends the 087 iotest to cover LUKS encryption when doing blockdev-add. Two further tests are added to validate read/write of LUKS encrypted images with a single file and with a backing file. Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz

[Qemu-block] [PATCH v9 13/20] qcow2: add support for LUKS encryption format

This adds support for using LUKS as an encryption format with the qcow2 file, using the new encrypt.format parameter to request "luks" format. e.g. # qemu-img create --object secret,data=123456,id=sec0 \ -f qcow2 -o encrypt.format=luks,encrypt.key-secret=sec0 \ test.qcow2 10G The

[Qemu-block] [PATCH v9 07/20] block: deprecate "encryption=on" in favor of "encrypt.format=aes"

Historically the qcow & qcow2 image formats supported a property "encryption=on" to enable their built-in AES encryption. We'll soon be supporting LUKS for qcow2, so need a more general purpose way to enable encryption, with a choice of formats. This introduces an "encrypt.format" option, which

[Qemu-block] [PATCH v9 12/20] qcow2: extend specification to cover LUKS encryption

Update the qcow2 specification to describe how the LUKS header is placed inside a qcow2 file, when using LUKS encryption for the qcow2 payload instead of the legacy AES-CBC encryption Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Reviewed-by: Max Reitz

[Qemu-block] [PATCH v9 10/20] qcow2: make qcow2_encrypt_sectors encrypt in place

Instead of requiring separate input/output buffers for encrypting data, change qcow2_encrypt_sectors() to assume use of a single buffer, encrypting in place. The current callers all used the same buffer for input/output already. Reviewed-by: Eric Blake Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v9 08/20] qcow: make encrypt_sectors encrypt in place

Instead of requiring separate input/output buffers for encrypting data, change encrypt_sectors() to assume use of a single buffer, encrypting in place. One current caller uses the same buffer for input/output already and the other two callers are easily converted to do so. Reviewed-by: Alberto

[Qemu-block] [PATCH v9 03/20] qcow: document another weakness of qcow AES encryption

Document that use of guest virtual sector numbers as the basis for the initialization vectors is a potential weakness, when combined with internal snapshots or multiple images using the same passphrase. This fixes the formatting of the itemized list too. Reviewed-by: Max Reitz

[Qemu-block] [PATCH v9 04/20] qcow: require image size to be > 1 for new images

The qcow driver refuses to open images which are less than 2 bytes in size, but will happily create such images. Add a check in the create path to avoid this discrepancy. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake

[Qemu-block] [PATCH v9 01/20] block: expose crypto option names / defs to other drivers

The block/crypto.c defines a set of QemuOpts that provide parameters for encryption. This will also be needed by the qcow/qcow2 integration, so expose the relevant pieces in a new block/crypto.h header. Some helper methods taking QemuOpts are changed to take QDict to simplify usage in other

[Qemu-block] [PATCH v9 09/20] qcow: convert QCow to use QCryptoBlock for encryption

This converts the qcow driver to make use of the QCryptoBlock APIs for encrypting image content. This is only wired up to permit use of the legacy QCow encryption format. Users who wish to have the strong LUKS format should switch to qcow2 instead. With this change it is now required to use the

[Qemu-block] [PATCH v9 06/20] iotests: skip 048 with qcow which doesn't support resize

Test 048 is designed to verify data preservation during an image resize. The qcow (v1) format impl has never supported resize so always fails. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange ---

[Qemu-block] [PATCH v9 02/20] block: add ability to set a prefix for opt names

When integrating the crypto support with qcow/qcow2, we don't want to use the bare LUKS option names "hash-alg", "key-secret", etc. We need to namespace them to match the nested QAPI schema. e.g. "encrypt.hash-alg", "encrypt.key-secret" so that they don't clash with any general qcow options at a

[Qemu-block] [PATCH v9 05/20] iotests: skip 042 with qcow which dosn't support zero sized images

Test 042 is designed to verify operation with zero sized images. Such images are not supported with qcow (v1), so this test has always failed. Reviewed-by: Max Reitz Reviewed-by: Alberto Garcia Signed-off-by: Daniel P. Berrange ---

[Qemu-block] [PATCH v9 00/20] Convert QCow[2] to QCryptoBlock & add LUKS support

Previously posted: v1: https://lists.gnu.org/archive/html/qemu-devel/2017-01/msg00201.html v2: https://lists.gnu.org/archive/html/qemu-devel/2017-01/msg05147.html v3: https://lists.gnu.org/archive/html/qemu-devel/2017-01/msg05671.html v4:

Re: [Qemu-block] [PATCH v2 00/31] qed: Convert to coroutines

On Fri, Jun 16, 2017 at 07:36:45PM +0200, Kevin Wolf wrote: > The qed block driver is one of the last remaining block drivers that use the > AIO callback style interfaces. This series converts it to the coroutine model > that other drivers are using and removes some AIO functions from the block >

Re: [Qemu-block] [Qemu-devel] [PATCH] qemu-img: don't shadow opts variable in img_dd()

On 2017-06-19 17:00, Stefan Hajnoczi wrote: > It's confusing when two different variables have the same name in one > function. > > Cc: Reda Sallahi > Signed-off-by: Stefan Hajnoczi > --- > qemu-img.c | 9 +++-- > 1 file changed, 3 insertions(+), 6

Re: [Qemu-block] [PATCH v4 00/16] block: Preallocated truncate

On Tue, Jun 13, 2017 at 10:20:51PM +0200, Max Reitz wrote: > === Series dependencies === > > This series depends on v7 of Stefan's series > "qemu-img: add measure sub-command" > (http://lists.nongnu.org/archive/html/qemu-devel/2017-06/msg03035.html). > > > === Actual cover letter === > > This

[Qemu-block] [PATCH] qemu-img: don't shadow opts variable in img_dd()

It's confusing when two different variables have the same name in one function. Cc: Reda Sallahi Signed-off-by: Stefan Hajnoczi --- qemu-img.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index

Re: [Qemu-block] [PATCH 2/4] qapi: Add qobject_is_equal()

On 2017-06-19 15:55, Kevin Wolf wrote: > Am 14.06.2017 um 17:31 hat Max Reitz geschrieben: >> This generic function (along with its implementations for different >> types) determines whether two QObjects are equal. >> >> Signed-off-by: Max Reitz > >> diff --git

Re: [Qemu-block] [PATCH 4/4] iotests: Add test for non-string option reopening

Am 14.06.2017 um 17:31 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH 3/4] block: qobject_is_equal() in bdrv_reopen_prepare()

Am 14.06.2017 um 17:31 hat Max Reitz geschrieben: > Currently, bdrv_reopen_prepare() assumes that all BDS options are > strings. However, this is not the case if the BDS has been created > through the json: pseudo-protocol or blockdev-add. > > Note that the user-invokable reopen command is an HMP

Re: [Qemu-block] [PATCH v8 19/20] qcow2: report encryption specific image information

On Wed, Jun 07, 2017 at 07:38:44PM +0200, Max Reitz wrote: > On 2017-06-01 19:27, Daniel P. Berrange wrote: > > Currently 'qemu-img info' reports a simple "encrypted: yes" > > field. This is not very useful now that qcow2 can support > > multiple encryption formats. Users want to know which format

Re: [Qemu-block] [PATCH v8 13/20] qcow2: add support for LUKS encryption format

On Wed, Jun 07, 2017 at 07:19:28PM +0200, Max Reitz wrote: > On 2017-06-01 19:27, Daniel P. Berrange wrote: > > This adds support for using LUKS as an encryption format > > with the qcow2 file, using the new encrypt.format parameter > > to request "luks" format. e.g. > > > > # qemu-img create

Re: [Qemu-block] [PATCH v8 09/20] qcow: convert QCow to use QCryptoBlock for encryption

On Wed, Jun 07, 2017 at 06:55:39PM +0200, Max Reitz wrote: > On 2017-06-01 19:27, Daniel P. Berrange wrote: > > This converts the qcow driver to make use of the QCryptoBlock > > APIs for encrypting image content. This is only wired up to > > permit use of the legacy QCow encryption format. Users

Re: [Qemu-block] [PATCH v8 07/20] block: deprecate "encryption=on" in favor of "encrypt.format=aes"

On Wed, Jun 07, 2017 at 06:40:32PM +0200, Max Reitz wrote: > On 2017-06-01 19:27, Daniel P. Berrange wrote: > > Historically the qcow & qcow2 image formats supported a property > > "encryption=on" to enable their built-in AES encryption. We'll > > soon be supporting LUKS for qcow2, so need a more

Re: [Qemu-block] [PATCH 2/4] qapi: Add qobject_is_equal()

Am 14.06.2017 um 17:31 hat Max Reitz geschrieben: > This generic function (along with its implementations for different > types) determines whether two QObjects are equal. > > Signed-off-by: Max Reitz > diff --git a/qobject/qdict.c b/qobject/qdict.c > index 88e2ecd..2ed57ca

[Qemu-block] [PATCH v4 6/7] qcow2: Pass a QEMUIOVector to do_perform_cow_{read, write}()

Instead of passing a single buffer pointer to do_perform_cow_write(), pass a QEMUIOVector. This will allow us to merge the write requests for the COW regions and the actual data into a single one. Although do_perform_cow_read() does not strictly need to change its API, we're doing it here as well

[Qemu-block] [PATCH v4 0/7] Reduce the number of I/O ops when doing COW

Hi all, here's a patch series that rewrites the copy-on-write code in the qcow2 driver to reduce the number of I/O operations. This is version v4, please refer to the original e-mail for a complete description: https://lists.gnu.org/archive/html/qemu-block/2017-05/msg00882.html Regards, Berto

[Qemu-block] [PATCH v4 4/7] qcow2: Split do_perform_cow() into _read(), _encrypt() and _write()

This patch splits do_perform_cow() into three separate functions to read, encrypt and write the COW regions. perform_cow() can now read both regions first, then encrypt them and finally write them to disk. The memory allocation is also done in this function now, using one single buffer large

[Qemu-block] [PATCH v4 7/7] qcow2: Merge the writing of the COW regions with the guest data

If the guest tries to write data that results on the allocation of a new cluster, instead of writing the guest data first and then the data from the COW regions, write everything together using one single I/O operation. This can improve the write performance by 25% or more, depending on several

[Qemu-block] [PATCH v4 2/7] qcow2: Use unsigned int for both members of Qcow2COWRegion

Qcow2COWRegion has two attributes: - The offset of the COW region from the start of the first cluster touched by the I/O request. Since it's always going to be positive and the maximum request size is at most INT_MAX, we can use a regular unsigned int to store this offset. - The size of

[Qemu-block] [PATCH v4 5/7] qcow2: Allow reading both COW regions with only one request

Reading both COW regions requires two separate requests, but it's perfectly possible to merge them and perform only one. This generally improves performance, particularly on rotating disk drives. The downside is that the data in the middle region is read but discarded. This patch takes a

[Qemu-block] [PATCH v4 3/7] qcow2: Make perform_cow() call do_perform_cow() twice

Instead of calling perform_cow() twice with a different COW region each time, call it just once and make perform_cow() handle both regions. This patch simply moves code around. The next one will do the actual reordering of the COW operations. Signed-off-by: Alberto Garcia

[Qemu-block] [PATCH v4 1/7] qcow2: Remove unused Error variable in do_perform_cow()

We are using the return value of qcow2_encrypt_sectors() to detect problems but we are throwing away the returned Error since we have no way to report it to the user. Therefore we can simply get rid of the local Error variable and pass NULL instead. Alternatively we could try to figure out a way

Re: [Qemu-block] [PATCH 1/4] qapi/qnull: Add own header

Am 14.06.2017 um 17:30 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v2 7/7] qcow2: Merge the writing of the COW regions with the guest data

On Fri 16 Jun 2017 05:31:42 PM CEST, Kevin Wolf wrote: >> +/* Make sure that adding both COW regions to the QEMUIOVector >> + * does not exceed IOV_MAX */ >> +if (hd_qiov->niov > IOV_MAX - 2) { >> +continue; >> +} >> + >> +m->data_qiov = hd_qiov;

Re: [Qemu-block] [PATCH v2] live-block-ops.txt: Rename, rewrite, and improve it

On Fri, Jun 16, 2017 at 04:41:20PM +0100, Stephen Finucane wrote: > On Fri, 2017-06-16 at 16:51 +0200, Kashyap Chamarthy wrote: [...] > As requested, a couple of rST pointers below that will help you if/when you > switch to Sphinx. I've only focused on the design aspect, not the content. Thanks

Re: [Qemu-block] [Question] How can we confirm hot-plug disk succesfully?

Am 19.06.2017 um 12:27 hat Xie Changlong geschrieben: > 在 6/19/2017 3:27 PM, Kevin Wolf 写道: > >Am 18.06.2017 um 09:21 hat Xie Changlong geschrieben: > >>In device hot-remove scenario, if we don't probe acpiphp module on > >>the guest, 'device_del' will never emit DEVICE_DELETED event(because >

Re: [Qemu-block] [Question] How can we confirm hot-plug disk succesfully?

在 6/19/2017 3:27 PM, Kevin Wolf 写道: Am 18.06.2017 um 09:21 hat Xie Changlong geschrieben: In device hot-remove scenario, if we don't probe acpiphp module on the guest, 'device_del' will never emit DEVICE_DELETED event(because guest will not write to __EJ0) . So we can confirm that hot-remove

Re: [Qemu-block] [Question] How can we confirm hot-plug disk succesfully?

Am 18.06.2017 um 09:21 hat Xie Changlong geschrieben: > In device hot-remove scenario, if we don't probe acpiphp module on > the guest, 'device_del' will never emit DEVICE_DELETED event(because > guest will not write to __EJ0) . So we can confirm that hot-remove > failed. But IIUC, there is no