[Qemu-block] [PATCH v10 1/2] mirror: Rewrite mirror_iteration

The "pnum < nb_sectors" condition in deciding whether to actually copy data is unnecessarily strict, and the qiov initialization is unnecessarily for bdrv_aio_write_zeroes and bdrv_aio_discard. Rewrite mirror_iteration to fix both flaws. The output of iotests 109 is updated because we now report

[Qemu-block] [PATCH v10 0/2] mirror: Improve zero write and discard

v10: Fix and simplify mirror_cow_align. [Max] v9: Fix the one bug Max found: "shrink *= chunk_sectors". v8: Rebase onto master (didn't pick up Max's rev-by due to non-trivial code change). The conflict is around removed lines about "max_iov" and "IOV_MAX" due to commit

Re: [Qemu-block] [Qemu-devel] [PATCH v1 01/15] crypto: add cryptographic random byte source

On Tue, 01/12 18:56, Daniel P. Berrange wrote: > Signed-off-by: Daniel P. Berrange > --- > crypto/Makefile.objs| 1 + > crypto/random.c | 50 > + > include/crypto/random.h | 43

Re: [Qemu-block] [Qemu-devel] [PATCH v1 02/15] crypto: add support for PBKDF2 algorithm

On Tue, 01/12 18:56, Daniel P. Berrange wrote: > +#if defined CONFIG_NETTLE > +#include "crypto/pbkdf-nettle.c" > +#elif defined CONFIG_GCRYPT > +#include "crypto/pbkdf-gcrypt.c" > +#else /* ! CONFIG_GCRYPT */ > +#include "crypto/pbkdf-stub.c" > +#endif /* ! CONFIG_GCRYPT */ I think the

Re: [Qemu-block] [Qemu-devel] [PATCH 06/13] HBitmap: Introduce "meta" bitmap to track bit changes

On 11.01.2016 21:56, John Snow wrote: On 01/11/2016 10:40 AM, Vladimir Sementsov-Ogievskiy wrote: On 04.01.2016 13:27, Fam Zheng wrote: Upon each bit toggle, the corresponding bit in the meta bitmap will be set. Signed-off-by: Fam Zheng --- include/qemu/hbitmap.h | 8

Re: [Qemu-block] [PATCH 4/5] block/backup: Add subclassed notifier

On 12/01/2016 01:36, John Snow wrote: > Instead of relying on peeking at bs->job, we want to explicitly get > a reference to the job that was involved in this notifier callback. > > Extend the Notifier to include a job pointer, and include a reference > to the job registering the callback. This

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 07:03 hat Denis V. Lunev geschrieben: > There is a long-long story. OVMF VMs can not be snapsotted using > 'virsh snapshot' as they have "pflash" device which is configured as > "raw" image. There was a discussion in the past about that. > > Good description has been provided on

[Qemu-block] [PATCH v1 07/15] block: add flag to indicate that no I/O will be performed

When opening an image it is useful to know whether the caller intends to perform I/O on the image or not. In the case of encrypted images this will allow the block driver to avoid having to prompt for decryption keys when we merely want to query header metadata about the image. eg qemu-img info

[Qemu-block] [PATCH v1 13/15] block: rip out all traces of password prompting

Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, nothing is relying on the interactive prompting for passwords. All the code related to password prompting can thus be ripped out. Signed-off-by: Daniel P. Berrange --- hmp.c

[Qemu-block] [PATCH v1 03/15] crypto: add support for generating initialization vectors

There are a number of different algorithms that can be used to generate initialization vectors for disk encryption. This introduces a simple internal QCryptoBlockIV object to provide a consistent internal API to the different algorithms. The initially implemented algorithms are 'plain', 'plain64'

[Qemu-block] [PATCH v1 02/15] crypto: add support for PBKDF2 algorithm

The LUKS data format includes use of PBKDF2 (Password-Based Key Derivation Function). The Nettle library can provide an implementation of this, but we don't want code directly depending on a specific crypto library backend. Introduce a include/crypto/pbkdf.h header which defines a QEMU API for

[Qemu-block] [PATCH v1 14/15] block: remove all encryption handling APIs

Now that all encryption keys must be provided upfront via the QCryptoSecret API and associated block driver properties there is no need for any explicit encryption handling APIs in the block layer. Encryption can be handled transparently within the block driver. We only retain an API for querying

[Qemu-block] [PATCH v1 09/15] qcow2: make qcow2_encrypt_sectors encrypt in place

Instead of requiring separate input/output buffers for encrypting data, change qcow2_encrypt_sectors() to assume use of a single buffer, encrypting in place. The current callers all used the same buffer for input/output already. Signed-off-by: Daniel P. Berrange ---

Re: [Qemu-block] [Qemu-devel] [PATCH 4/5] block/backup: Add subclassed notifier

On 01/12/2016 01:01 PM, Paolo Bonzini wrote: > > > On 12/01/2016 18:57, John Snow wrote: >> >> >> On 01/12/2016 03:46 AM, Paolo Bonzini wrote: >>> >>> >>> On 12/01/2016 01:36, John Snow wrote: Instead of relying on peeking at bs->job, we want to explicitly get a reference to the job

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 18:40 hat Markus Armbruster geschrieben: > Kevin Wolf writes: > > > Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: > >> On 01/12/2016 06:47 PM, Denis V. Lunev wrote: > >> >On 01/12/2016 06:20 PM, Kevin Wolf wrote: > >> >>Am 12.01.2016 um 15:59 hat Paolo

Re: [Qemu-block] [Qemu-devel] [PATCH 4/5] block/backup: Add subclassed notifier

On 12/01/2016 18:57, John Snow wrote: > > > On 01/12/2016 03:46 AM, Paolo Bonzini wrote: >> >> >> On 12/01/2016 01:36, John Snow wrote: >>> Instead of relying on peeking at bs->job, we want to explicitly get >>> a reference to the job that was involved in this notifier callback. >>> >>> Extend

[Qemu-block] [PATCH v1 15/15] block: remove support for legecy AES qcow/qcow2 encryption

Refuse to use images with the legacy AES-CBC encryption format in the system emulators. They are still fully supported in the qemu-img, qemu-io & qemu-nbd tools in order to allow data to be liberated and for compatibility with older QEMU versions. Continued support in these tools is not a notable

[Qemu-block] [PATCH v1 12/15] qcow: convert QCow to use QCryptoBlock for encryption

This converts the qcow2 driver to make use of the QCryptoBlock APIs for encrypting image content. This is only wired up to permit use of the legacy QCow encryption format. Users who wish to have the strong LUKS format should switch to qcow2 instead. With this change it is now required to use the

[Qemu-block] [PATCH v1 00/15] Support LUKS encryption in block devices

This is a posting of the previously submitted work in progress code: https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg04748.html This series depends on these previously submitted patches to the block tools: https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg04354.html As can

[Qemu-block] [PATCH v1 01/15] crypto: add cryptographic random byte source

Signed-off-by: Daniel P. Berrange --- crypto/Makefile.objs| 1 + crypto/random.c | 50 + include/crypto/random.h | 43 ++ 3 files changed, 94 insertions(+) create mode

[Qemu-block] [PATCH v1 11/15] qcow: make encrypt_sectors encrypt in place

Instead of requiring separate input/output buffers for encrypting data, change encrypt_sectors() to assume use of a single buffer, encrypting in place. One current caller all uses the same buffer for input/output already and the other two callers are easily converted todo so. Signed-off-by:

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 08:40 PM, Markus Armbruster wrote: Kevin Wolf writes: Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: On 01/12/2016 06:47 PM, Denis V. Lunev wrote: On 01/12/2016 06:20 PM, Kevin Wolf wrote: Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: On

Re: [Qemu-block] [Qemu-devel] [PATCH 4/5] block/backup: Add subclassed notifier

On 01/12/2016 03:46 AM, Paolo Bonzini wrote: > > > On 12/01/2016 01:36, John Snow wrote: >> Instead of relying on peeking at bs->job, we want to explicitly get >> a reference to the job that was involved in this notifier callback. >> >> Extend the Notifier to include a job pointer, and include

[Qemu-block] [PATCH v1 08/15] block: add generic full disk encryption driver

Add a block driver that is capable of supporting any full disk encryption format. This utilizes the previously added block encryption code, and at this time supports the LUKS format. The driver code is capable of supporting any format supported by the QCryptoBlock module, so it registers one

[Qemu-block] [PATCH v1 04/15] crypto: add support for anti-forensic split algorithm

The LUKS format specifies an anti-forensic split algorithm which is used to artificially expand the size of the key material on disk. This is an implementation of that algorithm. Signed-off-by: Daniel P. Berrange --- crypto/Makefile.objs| 1 + crypto/afsplit.c

[Qemu-block] [PATCH v1 10/15] qcow2: convert QCow2 to use QCryptoBlock for encryption

This converts the qcow2 driver to make use of the QCryptoBlock APIs for encrypting image content. As well as continued support for the legacy QCow2 encryption format, the appealing benefit is that it enables support for the LUKS format inside qcow2. With the LUKS format it is neccessary to store

[Qemu-block] [PATCH v1 06/15] crypto: implement the LUKS block encryption format

Provide a block encryption implementation that follows the LUKS/dm-crypt specification. This supports all combinations of hash, cipher algorithm, cipher mode and iv generator that are implemented by the current crypto layer. The notable missing feature is support for the 'xts' cipher mode, which

Re: [Qemu-block] [PATCH v2 2/2] nbd: Coroutine based nbd_send_negotiate

On Mon, 01/11 15:00, Paolo Bonzini wrote: > > > On 11/01/2016 04:36, Fam Zheng wrote: > > > > +aio_set_fd_handler(ctx, client->sock, true, > > + nbd_negotiate_continue, > > + nbd_negotiate_continue, data->co); > > TRACE("Beginning

Re: [Qemu-block] [PATCH v9 1/2] mirror: Rewrite mirror_iteration

On Wed, 01/06 18:53, Max Reitz wrote: > On 05.01.2016 09:46, Fam Zheng wrote: > > The "pnum < nb_sectors" condition in deciding whether to actually copy > > data is unnecessarily strict, and the qiov initialization is > > unnecessarily for bdrv_aio_write_zeroes and bdrv_aio_discard. > > > >

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 16:10 hat Denis V. Lunev geschrieben: > On 01/12/2016 05:16 PM, Kevin Wolf wrote: > >Am 12.01.2016 um 07:03 hat Denis V. Lunev geschrieben: > >>There is a long-long story. OVMF VMs can not be snapsotted using > >>'virsh snapshot' as they have "pflash" device which is configured as

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 12 January 2016 at 15:13, Denis V. Lunev wrote: > The idea of this patch was trivial. First of all, I would like to keep > this image internally snapshoted. That is why the ultimate goal > was to switch from raw to qcow2 to keep changes inside the > image. > > Though in this

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: > > > On 12/01/2016 15:16, Kevin Wolf wrote: > >> Thus we should avoid selection of "pflash" drives for VM state saving. > >> > >> For now "pflash" is read-write raw image as it configured by libvirt. > >> Thus there are no such images in the

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 16:16 hat Peter Maydell geschrieben: > On 12 January 2016 at 15:13, Denis V. Lunev wrote: > > The idea of this patch was trivial. First of all, I would like to keep > > this image internally snapshoted. That is why the ultimate goal > > was to switch from raw to

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 12/01/2016 15:16, Kevin Wolf wrote: >> Thus we should avoid selection of "pflash" drives for VM state saving. >> >> For now "pflash" is read-write raw image as it configured by libvirt. >> Thus there are no such images in the field and we could safely disable >> ability to save state to those

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 05:59 PM, Paolo Bonzini wrote: On 12/01/2016 15:16, Kevin Wolf wrote: Thus we should avoid selection of "pflash" drives for VM state saving. For now "pflash" is read-write raw image as it configured by libvirt. Thus there are no such images in the field and we could safely

Re: [Qemu-block] [Qemu-devel] [PATCH 06/10] qemu-img: Prepare for locked images

Kevin Wolf writes: > Am 11.01.2016 um 16:49 hat Markus Armbruster geschrieben: >> Eric Blake writes: >> >> > On 12/22/2015 09:46 AM, Kevin Wolf wrote: >> >> This patch extends qemu-img for working with locked images. It prints a >> >> helpful error message

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 05:16 PM, Kevin Wolf wrote: Am 12.01.2016 um 07:03 hat Denis V. Lunev geschrieben: There is a long-long story. OVMF VMs can not be snapsotted using 'virsh snapshot' as they have "pflash" device which is configured as "raw" image. There was a discussion in the past about that.

Re: [Qemu-block] [PATCH 1/2] blockdev: Error out on negative throttling option values

On Mon 11 Jan 2016 06:42:38 AM CET, Fam Zheng wrote: > The implicit casting from unsigned int to double changes negative > values into large positive numbers, whereas explicitly casting to > signed integer first will let us catch the invalid value and report > error correctly: >

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 12/01/2016 16:20, Kevin Wolf wrote: > > Flash is already being snapshotted as part of saving RAM state. In > > fact, for this reason the device (at least the one used with OVMF; I > > haven't checked other pflash devices) can simply save it back to disk > > on the migration destination,

[Qemu-block] [PATCH 2/4] blockdev: Fix 'change' for slot devices

'change' and related operations did not work when used on guest devices featuring removable media but no actual tray, because blk_dev_is_tray_open() always returned false for them and the blockdev-{insert,remove}-medium commands required it to return true. Fix this by making

[Qemu-block] [PATCH 1/4] block: Add blk_dev_has_tray()

Pull out the check whether a block device has a tray from blk_dev_is_tray_open() into an own function so both attributes (whether there is a tray vs. whether that tray is open) can be queried independently. Cc: qemu-stable Signed-off-by: Max Reitz ---

[Qemu-block] [PATCH 4/4] block/qapi: Emit tray_open only if there is a tray

Signed-off-by: Max Reitz --- block/qapi.c | 2 +- tests/qemu-iotests/067.out | 4 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index 58d3975..12a0f25 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -299,7 +299,7

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 06:20 PM, Kevin Wolf wrote: Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: On 12/01/2016 15:16, Kevin Wolf wrote: Thus we should avoid selection of "pflash" drives for VM state saving. For now "pflash" is read-write raw image as it configured by libvirt. Thus there are

[Qemu-block] [PATCH 0/4] blockdev: Fix 'change' for slot devices

The series "BlockBackend and media" intended all block devices with removable media to implement a tray model; if the devices does not have a tray, it should emulate one. While this may make sense from a technical perspective (blockdev-*-tray are guest device controlling operations, invoking

[Qemu-block] [PATCH 3/4] Revert "hw/block/fdc: Implement tray status"

This reverts the changes commit 2e1280e8ff95b3145bc6262accc9d447718e5318 applied to hw/block/fdc.c. That commit changed tests/fdc-test.c, too, because after it, one less TRAY_MOVED event would be emitted when executing 'change' on an empty drive. However, now, no TRAY_MOVED events will be emitted

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 08:50 PM, Kevin Wolf wrote: Am 12.01.2016 um 18:40 hat Markus Armbruster geschrieben: Kevin Wolf writes: Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: On 01/12/2016 06:47 PM, Denis V. Lunev wrote: On 01/12/2016 06:20 PM, Kevin Wolf wrote: Am

[Qemu-block] [PATCH v1 05/15] crypto: add block encryption framework

Add a generic framework for support different block encryption formats. Upon instantiating a QCryptoBlock object, it will read the encryption header and extract the encryption keys. It is then possible to call methods to encrypt/decrypt data buffers. There is also a mode whereby it will

[Qemu-block] [PATCH v2 0/2] block: Reject negative values for throttling options

v2: Check the value range and report an appropriate error. [Berto] Now the negative values are silently converted to a huge positive number because we are doing implicit casting from uint64_t to double. Fix it and add a test case (this was once fixed in 7d81c1413c9 but regressed when the block

[Qemu-block] [PATCH v2 1/2] blockdev: Error out on negative throttling option values

The implicit casting from unsigned int to double changes negative values into large positive numbers and accepts them. We should instead print an error. Check the number range so this case is catched and reported. Signed-off-by: Fam Zheng --- blockdev.c | 3 ++-

[Qemu-block] [PATCH v2 2/2] iotests: Test that negative throttle values are rejected

Signed-off-by: Fam Zheng --- tests/qemu-iotests/051| 11 +++ tests/qemu-iotests/051.out| 21 + tests/qemu-iotests/051.pc.out | 21 + 3 files changed, 53 insertions(+) diff --git a/tests/qemu-iotests/051

Re: [Qemu-block] [PATCH 1/2] blockdev: Error out on negative throttling option values

On Tue, 01/12 16:00, Alberto Garcia wrote: > On Mon 11 Jan 2016 06:42:38 AM CET, Fam Zheng wrote: > > > The implicit casting from unsigned int to double changes negative > > values into large positive numbers, whereas explicitly casting to > > signed integer first will let us

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 07:52 PM, Kevin Wolf wrote: Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: On 01/12/2016 06:47 PM, Denis V. Lunev wrote: On 01/12/2016 06:20 PM, Kevin Wolf wrote: Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: On 12/01/2016 15:16, Kevin Wolf wrote: Thus we

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

On 01/12/2016 06:47 PM, Denis V. Lunev wrote: On 01/12/2016 06:20 PM, Kevin Wolf wrote: Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: On 12/01/2016 15:16, Kevin Wolf wrote: Thus we should avoid selection of "pflash" drives for VM state saving. For now "pflash" is read-write raw

Re: [Qemu-block] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: > On 01/12/2016 06:47 PM, Denis V. Lunev wrote: > >On 01/12/2016 06:20 PM, Kevin Wolf wrote: > >>Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: > >>> > >>>On 12/01/2016 15:16, Kevin Wolf wrote: > >Thus we should avoid selection of

Re: [Qemu-block] [Qemu-devel] [PATCH 06/10] qemu-img: Prepare for locked images

Am 12.01.2016 um 16:20 hat Markus Armbruster geschrieben: > Kevin Wolf writes: > > > Am 11.01.2016 um 16:49 hat Markus Armbruster geschrieben: > >> Eric Blake writes: > >> > >> > On 12/22/2015 09:46 AM, Kevin Wolf wrote: > >> >> This patch extends qemu-img

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] blk: do not select PFLASH device for internal snapshot

Kevin Wolf writes: > Am 12.01.2016 um 17:35 hat Denis V. Lunev geschrieben: >> On 01/12/2016 06:47 PM, Denis V. Lunev wrote: >> >On 01/12/2016 06:20 PM, Kevin Wolf wrote: >> >>Am 12.01.2016 um 15:59 hat Paolo Bonzini geschrieben: >> >>> >> >>>On 12/01/2016 15:16, Kevin Wolf