Re: [Qemu-block] [RHEL-7.2.z qemu-kvm-rhev PATCH] virtio-blk: Fix double completion for werror=stop

On 18/11/2015 04:19, Fam Zheng wrote: > BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1277922 > Brew: http://brewweb.devel.redhat.com/brew/taskinfo?taskID=10113752 > > When a request R is absorbed by request M, it is appended to the > "mr_next" queue led by M, and is completed together with

[Qemu-block] [PATCH] tests: fix cdrom_pio_impl in ide-test

The check for the cleared BSY flag has to be performed before each data transfer and not just before the first one. Commit 5f81724d revealed this glitch as the BSY flag was not set in ATAPI PIO transfers before. While at it fix the desciptions and add a comment before the nested for loop that

Re: [Qemu-block] [PATCH] tests: fix cdrom_pio_impl in ide-test

On 20 November 2015 at 14:29, Peter Lieven wrote: > The check for the cleared BSY flag has to be performed > before each data transfer and not just before the > first one. > > Commit 5f81724d revealed this glitch as the BSY flag > was not set in ATAPI PIO transfers before. > > While

[Qemu-block] [PATCH] qemu-iotests: Add -nographic when starting QEMU in 120

Otherwise, a window flashes on my desktop (built with SDL). Other iotest cases have that. Signed-off-by: Fam Zheng --- tests/qemu-iotests/120 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120 index

[Qemu-block] [PATCH for-2.5] block-migration: limit the memory usage

If we set migration speed in a very large value, block-migration will try to read all data to the memory. Because (block_mig_state.submitted + block_mig_state.read_done) * BLOCK_SIZE will be overflow, and it will be always less than rate limit. There is no need to read too many data into

Re: [Qemu-block] [PATCH 1/2] mirror: Rewrite mirror_iteration

On Wed, 11/18 11:38, Paolo Bonzini wrote: > > > On 17/11/2015 12:41, Fam Zheng wrote: > > +/* Wait for I/O to this cluster (from a previous iteration) to be > > + * done.*/ > > +while (test_bit(next_chunk, s->in_flight_bitmap)) { > > +

[Qemu-block] [PATCH for 2.6 3/3] hbitmap: Drop "granularity"

Sometimes confused with the granularity with coarse levels in HBitmap, the granularity in the hbitmap_alloc is not an essential concept of a bitmap. Now that all callers except the test code use zero, it's possible to drop the parameter to make the interface cleaner and more intuitive. Test code

[Qemu-block] [PATCH v6 1/2] mirror: Rewrite mirror_iteration

The "pnum < nb_sectors" condition in deciding whether to actually copy data is unnecessarily strict, and the qiov initialization is unnecessarily for bdrv_aio_write_zeroes and bdrv_aio_discard. Rewrite mirror_iteration to fix both flaws. Signed-off-by: Fam Zheng --- v5:

[Qemu-block] [PATCH v6 2/2] mirror: Add mirror_wait_for_io

The three lines are duplicated a number of times now, refactor a function. Signed-off-by: Fam Zheng --- block/mirror.c | 24 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index ff8149d..ea5a76a 100644

[Qemu-block] [PATCH v6 0/2] mirror: Improve zero write and discard

v6: Address Paolo's comments in mirror_iteration: - break if we've already got a chunk to work on; - fix a typo in comment. Patch 1 rewrites mirror_iteration. Patch 2 is a small DRY cleaning up. Fam Zheng (2): mirror: Rewrite mirror_iteration mirror: Add mirror_wait_for_io

[Qemu-block] [PATCH] block/qapi: Plug memory leak on query-block error path

Spotted by Coverity. Signed-off-by: Markus Armbruster --- block/qapi.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/block/qapi.c b/block/qapi.c index d20262d..267f147 100644 --- a/block/qapi.c +++ b/block/qapi.c @@ -436,7 +436,9 @@ BlockInfoList

[Qemu-block] [PATCH for 2.6 1/3] backup: Use Bitmap to replace "s->bitmap"

"s->bitmap" tracks done sectors, we only check bit states without using any iterator which HBitmap is good for. Switch to "Bitmap" which is simpler and more memory efficient. Meanwhile, rename it to done_bitmap, to reflect the intention. Signed-off-by: Fam Zheng ---

[Qemu-block] [PATCH for 2.6 2/3] block: Hide HBitmap in block dirty bitmap interface

HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before, an HBitmapIter is initialized in place, now the new BdrvDirtyBitmapIter must be

[Qemu-block] [PATCH for 2.6 0/3] Bitmap clean-up patches for 2.6

This makes a cleaner base for more dirty bitmap work. "granularity" appearing with different representations have always been mind twisting, remove it from HBitmap to make the interface and implementation simpler. Upon this, it is a bit easier to add persistent dirty bitmap functionalities. Block

Re: [Qemu-block] [Qemu-devel] [PATCH] tests: fix cdrom_pio_impl in ide-test

On 11/20/2015 09:37 AM, Peter Maydell wrote: > On 20 November 2015 at 14:29, Peter Lieven wrote: >> The check for the cleared BSY flag has to be performed >> before each data transfer and not just before the >> first one. >> >> Commit 5f81724d revealed this glitch as the BSY flag

Re: [Qemu-block] [PATCH for 2.6 1/3] backup: Use Bitmap to replace "s->bitmap"

On 20.11.2015 12:59, Fam Zheng wrote: "s->bitmap" tracks done sectors, we only check bit states without using any iterator which HBitmap is good for. Switch to "Bitmap" which is simpler and more memory efficient. Meanwhile, rename it to done_bitmap, to reflect the intention. Signed-off-by: Fam

Re: [Qemu-block] [PATCH for 2.6 3/3] hbitmap: Drop "granularity"

On 20.11.2015 12:59, Fam Zheng wrote: Sometimes confused with the granularity with coarse levels in HBitmap, the granularity in the hbitmap_alloc is not an essential concept of a bitmap. Now that all callers except the test code use zero, it's possible to drop the parameter to make the

Re: [Qemu-block] [PATCH v5] raw-posix.c: Make physical devices usable in QEMU under Mac OS X host

Am 27.07.2015 um 19:05 hat Programmingkid geschrieben: > Mac OS X can be picky when it comes to allowing the user to use physical > devices > in QEMU. Most mounted volumes appear to be off limits to QEMU. If an issue is > detected, a message is displayed showing the user how to unmount a volume. >

Re: [Qemu-block] [PATCH for 2.6 2/3] block: Hide HBitmap in block dirty bitmap interface

On 20.11.2015 12:59, Fam Zheng wrote: HBitmap is an implementation detail of block dirty bitmap that should be hidden from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying HBitmapIter. A small difference in the interface is, before, an HBitmapIter is initialized in place,

Re: [Qemu-block] [PATCH for 2.6 0/3] Bitmap clean-up patches for 2.6

Hi Fam! Thanks for it, I really like the idea about dropping granularity from hbitmap. I've waste lots of time understanding the code about bitmaps. Keeping in mind what units are in what granularity and what granularity is in what units is more than inconvenient) On 20.11.2015 12:59, Fam

Re: [Qemu-block] [PATCH] tests: fix cdrom_pio_impl in ide-test

Am 20.11.2015 um 15:29 hat Peter Lieven geschrieben: > The check for the cleared BSY flag has to be performed > before each data transfer and not just before the > first one. > > Commit 5f81724d revealed this glitch as the BSY flag > was not set in ATAPI PIO transfers before. > > While at it fix

Re: [Qemu-block] [PATCH] qemu-iotests: Add -nographic when starting QEMU in 120

On 20.11.2015 10:35, Fam Zheng wrote: > Otherwise, a window flashes on my desktop (built with SDL). Other > iotest cases have that. > > Signed-off-by: Fam Zheng > --- > tests/qemu-iotests/120 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.5] block/qapi: Plug memory leak on query-block error path

On 11/20/2015 05:53 AM, Markus Armbruster wrote: > Spotted by Coverity. > > Signed-off-by: Markus Armbruster > --- > block/qapi.c | 8 +++- > 1 file changed, 3 insertions(+), 5 deletions(-) > > diff --git a/block/qapi.c b/block/qapi.c > index d20262d..267f147 100644 >

[Qemu-block] [PATCH WIP 04/30] qcow2: add a 'keyid' parameter to qcow2 options

Add a 'keyid' parameter that refers to the ID of a QCryptoSecret instance that provides the encryption key. $QEMU \ -object secret,id=sec0,filename=/home/berrange/encrypted.pw \ -drive file=/home/berrange/encrypted.qcow2,keyid=sec0 Signed-off-by: Daniel P. Berrange

[Qemu-block] [PATCH WIP 07/30] qemu-nbd: add support for --object command line arg

Allow creation of user creatable object types with qemu-nbd via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # echo -n letmein > mypasswd.txt # qemu-nbd --object

[Qemu-block] [PATCH WIP 02/30] crypto: add support for loading encrypted x509 keys

Make use of the QCryptoSecret object to support loading of encrypted x509 keys. The optional 'passwordid' parameter to the tls-creds-x509 object type, provides the ID of a secret object instance that holds the decryption password for the PEM file. # echo "123456" > mypasswd.txt # $QEMU \

[Qemu-block] [PATCH WIP 05/30] qom: add user_creatable_add & user_creatable_del methods

The QMP monitor code has two helper methods object_add and qmp_object_del that are called from several places in the code (QMP, HMP and main emulator startup). We soon need to use this code from qemu-img, qemu-io and qemu-nbd too, but don't want those to depend on the monitor. To avoid this,

[Qemu-block] [PATCH WIP 23/30] crypto: add support for generating initialization vectors

There are a number of different algorithms that can be used to generate initialization vectors for disk encryption. This introduces a simple internal QCryptoBlockIV object to provide a consistent internal API to the different algorithms. The initially implemented algorithms are 'plain', 'plain64'

[Qemu-block] [PATCH WIP 24/30] crypto: add support for anti-forensic split algorithm

The LUKS format specifies an anti-forensic split algorithm which is used to artificially expand the size of the key material on disk. This is an implementation of that algorithm. Signed-off-by: Daniel P. Berrange --- crypto/Makefile.objs| 1 + crypto/afsplit.c

[Qemu-block] [PATCH WIP 22/30] crypto: add support for PBKDF2 algorithm

The LUKS data format includes use of PBKDF2 (Password-Based Key Derivation Function). The Nettle library can provide an implementation of this, but we don't want code directly depending on a specific crypto library backend. Introduce a include/crypto/pbkdf.h header which defines a QEMU API for

Re: [Qemu-block] [Qemu-devel] [PATCH] tests: fix cdrom_pio_impl in ide-test

On 20 November 2015 at 17:12, John Snow wrote: > This looks correct. This will definitely fix the race in the test, since > it was due to a race where we were reading the data when DRQ was not set. > > Where I still remain a little confused is the precise flow control that >

Re: [Qemu-block] [Qemu-devel] [PATCH] block/qapi: Plug memory leak on query-block error path

On 11/20/2015 05:53 AM, Markus Armbruster wrote: > Spotted by Coverity. Worth mentioning that commit 553a7e87 was the culprit. > > Signed-off-by: Markus Armbruster > --- > block/qapi.c | 8 +++- > 1 file changed, 3 insertions(+), 5 deletions(-) > -- Eric Blake

[Qemu-block] [PATCH WIP 03/30] qcow: add a 'keyid' parameter to qcow options

Add a 'keyid' parameter that refers to the ID of a QCryptoSecret instance that provides the encryption key. eg $QEMU \ -object secret,id=sec0,filename=/home/berrange/encrypted.pw \ -drive file=/home/berrange/encrypted.qcow,keyid=sec0 Signed-off-by: Daniel P. Berrange

[Qemu-block] [PATCH WIP 06/30] qemu-img: add support for --object command line arg

Allow creation of user creatable object types with qemu-img via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # echo -n letmein > mypasswd.txt # qemu-img info --object

[Qemu-block] [PATCH WIP 13/30] block: remove all encryption handling APIs

Now that all encryption keys must be provided upfront via the QCryptoSecret API and associated block driver properties there is no need for any explicit encryption handling APIs in the block layer. Encryption can be handled transparently within the block driver. We only retain an API for querying

[Qemu-block] [PATCH WIP 01/30] crypto: add QCryptoSecret object class for password/key handling

Introduce a new QCryptoSecret object class which will be used for providing passwords and keys to other objects which need sensitive credentials. The new object can provide secret values directly as properties, or indirectly via a file. The latter includes support for file descriptor passing

[Qemu-block] [PATCH WIP 09/30] qemu-io: allow specifying image as a set of options args

Currently qemu-io allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg qemu-io https://127.0.0.1/images/centos7.iso qemu-io /home/berrange/demo.qcow2 This adds a --source arg (that is mutually exclusive with a positional

[Qemu-block] [PATCH v13 02/14] qobject: Rename qtype_code to QType

The name QType matches our CODING_STYLE conventions for type names in CamelCase. It also matches the fact that we are already naming all the enum members with a prefix of QTYPE, not QTYPE_CODE. And doing the rename will also make it easier for the next patch to use QAPI for providing the enum,

[Qemu-block] [PATCH WIP 28/30] block: add generic full disk encryption driver

Add a block driver that is capable of supporting any full disk encryption format. This utilizes the previously added block encryption code, and at this time supports the LUKS format. The driver code is capable of supporting any format supported by the QCryptoBlock module, so it registers one

[Qemu-block] [PATCH WIP 30/30] qcow2: add LUKS full disk encryption support

The QCow2 format currently has support for built-in AES encryption, however, this is fundamentally flawed from a cryptographic security POV, so its use is deprecated. The previously added generic full disk encryption driver could be used to encrypt QCow2 files by either laying it above or below

[Qemu-block] [PATCH WIP 12/30] block: rip out all traces of password prompting

Now that qcow & qcow2 are wired up to get encryption keys via the QCryptoSecret object, all traces of code which had to deal with prompting for passwords can be ripped out. When the image is initially opened, the encryption key must be available immediately, or an error will be reported. $

[Qemu-block] [PATCH WIP 14/30] block: remove support for writing to qcow/qcow2 encrypted images

Refuse to open a qcow/qcow2 image with encryption if write access has been requested. To enable historic data to be liberated support for reading images is retained, as it does not pose an unreasonable support burden now that the new key handling infrastructure is inplace. Signed-off-by: Daniel

[Qemu-block] [PATCH WIP 15/30] qcow2: make qcow2_encrypt_sectors encrypt in place

Instead of requiring separate input/output buffers for encrypting data, change qcow2_encrypt_sectors() to assume use of a single buffer, encrypting in place. The current callers all used the same buffer for input/output already. Signed-off-by: Daniel P. Berrange ---

[Qemu-block] [PATCH WIP 19/30] crypto: move QCryptoCipherAlgorithm/Mode enum definitions into QAPI

The QCryptoCipherAlgorithm and QCryptoCipherMode enums are defined in the crypto/cipher.h header. In the future some QAPI types will want to reference the hash enums, so move the enum definition into QAPI too. Signed-off-by: Daniel P. Berrange --- crypto/cipher.c |

[Qemu-block] [PATCH WIP 18/30] crypto: move QCryptoHashAlgorithm enum definition into QAPI

The QCryptoHashAlgorithm enum is defined in the crypto/hash.h header. In the future some QAPI types will want to reference the hash enums, so move the enum definition into QAPI too. Signed-off-by: Daniel P. Berrange --- crypto/hash.c | 4 ++-- include/crypto/hash.h

[Qemu-block] [PATCH WIP 26/30] crypto: add block encryption framework

Add a generic framework for support different block encryption formats. Upon instantiating a QCryptoBlock object, it will read the encryption header and extract the encryption keys. It is then possible to call methods to encrypt/decrypt data buffers. There is also a mode whereby it will

[Qemu-block] [PATCH WIP 11/30] qemu-img: allow specifying image as a set of options args

Currently qemu-img allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg qemu-img info https://127.0.0.1/images/centos7.iso This adds a --source arg (that is mutually exclusive with a positional filename arg and -f arg) that

[Qemu-block] [PATCH WIP 10/30] qemu-nbd: allow specifying image as a set of options args

Currently qemu-nbd allows an image filename to be passed on the command line, but does not have a way to set any options except the format eg qemu-nbd https://127.0.0.1/images/centos7.iso qemu-nbd /home/berrange/demo.qcow2 This adds a --source arg (that is mutually exclusive with a

[Qemu-block] [PATCH WIP 08/30] qemu-io: add support for --object command line arg

Allow creation of user creatable object types with qemu-io via a --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # echo -n letmein > mypasswd.txt # qemu-io --object

[Qemu-block] [PATCH WIP 20/30] crypto: ensure qapi/crypto.json is listed in qapi-modules

The rebuild of qapi-types.c/h is not correctly triggered when qapi/crypto.json is changed because it was missing from the list of files in the qapi-modules variable. Signed-off-by: Daniel P. Berrange --- Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff

[Qemu-block] [PATCH WIP 25/30] crypto: fix transposed arguments in cipher error message

When reporting an incorrect key length for a cipher, we mixed up the actual vs expected arguments. Signed-off-by: Daniel P. Berrange --- crypto/cipher.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/crypto/cipher.c b/crypto/cipher.c index

[Qemu-block] [PATCH WIP 16/30] crypto: add ability to query the cipher key, block & IV lens

Adds new methods to allow querying the length of the cipher key, block size and initialization vectors. Signed-off-by: Daniel P. Berrange --- crypto/cipher.c| 48 ++ include/crypto/cipher.h| 37

Re: [Qemu-block] [Qemu-devel] [PATCH] tests: fix cdrom_pio_impl in ide-test

On 11/20/2015 09:29 AM, Peter Lieven wrote: > The check for the cleared BSY flag has to be performed > before each data transfer and not just before the > first one. > > Commit 5f81724d revealed this glitch as the BSY flag > was not set in ATAPI PIO transfers before. > > While at it fix the

[Qemu-block] [PATCH WIP 27/30] crypto: implement the LUKS block encryption format

Provide a block encryption implementation that follows the LUKS/dm-crypt specification. This supports all combinations of hash, cipher algorithm, cipher mode and iv generator that are implemented by the current crypto layer. The notable missing feature is support for the 'xts' cipher mode, which

[Qemu-block] [PATCH WIP 29/30] qcow2: convert QCow2 to use QCryptoBlock for encryption

The QCryptoBlock framework trivially supports the legecy QCow encryption format. Convert QCow2 to use QCryptoBlock, since this will unlock the ability to support LUKS in QCow2 without increasing the code burden for encryption in QCow2. Signed-off-by: Daniel P. Berrange ---

Re: [Qemu-block] [Qemu-devel] [PATCH WIP 01/30] crypto: add QCryptoSecret object class for password/key handling

On 11/20/2015 11:04 AM, Daniel P. Berrange wrote: > Introduce a new QCryptoSecret object class which will be used > for providing passwords and keys to other objects which need > sensitive credentials. > > The new object can provide secret values directly as properties, > or indirectly via a

[Qemu-block] [PATCH v2] raw-posix.c: Make GetBSDPath() handle caching options

Add support for caching options that can be specified from the command line. Signed-off-by: John Arbuckle --- Only location of code has been changed. block/raw-posix.c | 15 +-- 1 files changed, 9 insertions(+), 6 deletions(-) diff --git

[Qemu-block] [PATCH v6] raw-posix.c: Make physical devices usable in QEMU under Mac OS X host

Mac OS X can be picky when it comes to allowing the user to use physical devices in QEMU. Most mounted volumes appear to be off limits to QEMU. If an issue is detected, a message is displayed showing the user how to unmount a volume. Signed-off-by: John Arbuckle ---

Re: [Qemu-block] [PATCH v5] raw-posix.c: Make physical devices usable in QEMU under Mac OS X host

On Nov 20, 2015, at 11:26 AM, Kevin Wolf wrote: > Am 27.07.2015 um 19:05 hat Programmingkid geschrieben: >> Mac OS X can be picky when it comes to allowing the user to use physical >> devices >> in QEMU. Most mounted volumes appear to be off limits to QEMU. If an issue is >> detected, a message