[Qemu-block] [PATCH v3 00/10] Make qemu-img/qemu-nbd/qemu-io CLI more flexible

This series of patches expands the syntax of the qemu-img, qemu-nbd and qemu-io commands to make them more flexible. v0: http://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html v1: https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg04014.html v2:

[Qemu-block] [PATCH v3 05/10] qemu-io: allow specifying image as a set of options args

Currently qemu-io allows an image filename to be passed on the command line, but unless using the JSON format, it does not have a way to set any options except the format eg qemu-io https://127.0.0.1/images/centos7.iso qemu-io /home/berrange/demo.qcow2 This adds a --image-opts arg that

[Qemu-block] [PATCH v3 02/10] qemu-img: add support for --object command line arg

Allow creation of user creatable object types with qemu-img via a new --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # printf letmein > mypasswd.txt # qemu-img info

[Qemu-block] [PATCH v3 03/10] qemu-nbd: add support for --object command line arg

Allow creation of user creatable object types with qemu-nbd via a new --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # printf letmein > mypasswd.txt # qemu-nbd --object

[Qemu-block] [PATCH v3 04/10] qemu-io: add support for --object command line arg

Allow creation of user creatable object types with qemu-io via a new --object command line arg. This will be used to supply passwords and/or encryption keys to the various block driver backends via the recently added 'secret' object type. # printf letmein > mypasswd.txt # qemu-io --object

[Qemu-block] [PATCH v3 07/10] qemu-img: allow specifying image as a set of options args

Currently qemu-img allows an image filename to be passed on the command line, but unless using the JSON format, it does not have a way to set any options except the format eg qemu-img info https://127.0.0.1/images/centos7.iso This adds a --image-opts arg that indicates that the positional

[Qemu-block] [PATCH v3 08/10] qemu-nbd: don't overlap long option values with short options

When defining values for long options, the normal practice is to start numbering from 256, to avoid overlap with the range of valid values for short options. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-nbd.c | 12 ++-- 1

[Qemu-block] [PATCH v3 06/10] qemu-nbd: allow specifying image as a set of options args

Currently qemu-nbd allows an image filename to be passed on the command line, but unless using the JSON format, it does not have a way to set any options except the format eg qemu-nbd https://127.0.0.1/images/centos7.iso qemu-nbd /home/berrange/demo.qcow2 This adds a --image-opts arg that

[Qemu-block] [PATCH v3 01/10] qom: add helpers for UserCreatable object types

The QMP monitor code has two helper methods object_add and qmp_object_del that are called from several places in the code (QMP, HMP and main emulator startup). The HMP and main emulator startup code also share further logic that extracts the qom-type & id values from a qdict. We soon need to use

[Qemu-block] [PATCH v3 10/10] qemu-io: use no_argument/required_argument constants

When declaring the 'struct option' array, use the standard constants no_argument/required_argument, instead of magic values 0 and 1. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-io.c | 30 +++--- 1 file

[Qemu-block] [PATCH v3 09/10] qemu-nbd: use no_argument/required_argument constants

When declaring the 'struct option' array, use the standard constants no_argument/required_argument, instead of magic values 0 and 1. Reviewed-by: Eric Blake Signed-off-by: Daniel P. Berrange --- qemu-nbd.c | 47

Re: [Qemu-block] [Qemu-devel] [PATCH v4 0/2] block: Reject negative values for throttling options

Fam Zheng writes: > v4: Add Max's rev-by in both patches, while fixing the "maxs" typo. > > v3: Address comments: > - Add test for large value; [Berto] > - Fix typos "negative" & "caught"; [Eric, Berto] > - Use "LL" suffix to the upper limit constant. [Berto] > > v2:

Re: [Qemu-block] [Qemu-devel] [PATCH v6 00/15] qemu-img map: Allow driver to return file of the allocated block

On Fri, 01/08 10:07, Fam Zheng wrote: > v6: Two more changes as suggested by Max: > - Fix comments of BDRV_BLOCK_OFFSET_VALID and BDRV_BLOCK_DATA; > - Really keep 102 working; > - Add rev-by in patch 11. Ping?

Re: [Qemu-block] [PATCH v14 0/8] Block replication for continuous checkpoints

ping... It seems that there is no feedback for a long time, we hope COLO prototype could be merged in QEMU 2.6, it depends on this series, so please help us. Thanks. zhanghailiang On 2016/1/14 9:12, Changlong Xie wrote: It seems i missed someone in CC list, add them. Thanks -Xie On

Re: [Qemu-block] [Qemu-devel] [PATCH v1 05/15] crypto: add block encryption framework

On Mon, Jan 18, 2016 at 12:48:56PM -0700, Eric Blake wrote: > On 01/14/2016 05:16 AM, Daniel P. Berrange wrote: > > >>> +# @qcowaes: QCow/QCow2 built-in AES-CBC encryption. Do not use > >>> +# > >> > >> Well, the only reason to use it would be to read data off an old > >> insecurely-encrypted

[Qemu-block] [PATCH] nbd: use client_close() when negotiate phase fails

When nbd_negotiate() fails, nbd_co_client_start() is directly calling client->close(). This eventually ends up calling nbd_client_put(), which does an assert(client->closing). Unfortunately we have not set the 'closing' flag, so the code now aborts. This bug was accidentally introduced in

Re: [Qemu-block] [PATCH v3 0/3] Use QCryptoSecret for block device passwords

On 19/01/2016 17:46, Daniel P. Berrange wrote: > On Tue, Jan 19, 2016 at 05:32:35PM +0100, Paolo Bonzini wrote: >> >> >> On 19/01/2016 14:51, Daniel P. Berrange wrote: >>> This series was previously posted: >>> >>> v1: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html >>>

[Qemu-block] [PATCH v13] block/raw-posix.c: Make physical devices usable in QEMU under Mac OS X host

Mac OS X can be picky when it comes to allowing the user to use physical devices in QEMU. Most mounted volumes appear to be off limits to QEMU. If an issue is detected, a message is displayed showing the user how to unmount a volume. Now QEMU uses both CD and DVD media. Signed-off-by: John

Re: [Qemu-block] [Qemu-devel] [PATCH 1/5] ide: Prohibit RESET on IDE drives

On 01/19/16 18:04, John Snow wrote: > > > On 01/19/2016 06:48 AM, Paolo Bonzini wrote: >> >> >> On 19/01/2016 05:51, John Snow wrote: >>> +/* Only RESET is allowed to an ATAPI device while BSY and/or DRQ are >>> set. */ >>> +if (s->status & (BUSY_STAT|DRQ_STAT)) { >>> +if (!(val

[Qemu-block] [PATCH v2 3/6] ide: move buffered DMA cancel to core

Buffered DMA cancellation was added to ATAPI devices and implemented for the BMDMA HBA. Move the code over to common IDE code and allow it to be used for any HBA. Signed-off-by: John Snow --- hw/ide/core.c | 45 +

[Qemu-block] [PATCH v2 6/6] ide: fix device_reset to not ignore pending AIO

Signed-off-by: John Snow --- hw/ide/core.c | 27 +-- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index 9bc8e58..c68d1d4 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -502,7 +502,6 @@ void

Re: [Qemu-block] [Qemu-devel] [PATCH] ide: Correct the CHS 'cyls_max' limit to be 65535

On 01/18/2016 05:19 AM, Shmulik Ladkani wrote: > In b7eb0c9: > hw/block-common: Factor out fall back to legacy -drive cyls=... > 'blkconf_geometry()' was introduced, factoring out CHS limit validation > code that was repeated in ide, scsi, virtio-blk. > > The original IDE CHS limit prior

Re: [Qemu-block] [PATCH v3 04/13] nbd: convert to using I/O channels for actual socket I/O

On 19/01/2016 17:52, Daniel P. Berrange wrote: > I'll have a look at this and see what I can do. It shouldn't block > merging of this patch I assume, since the current call to > qemu_coroutine_yield() should be fine - it just my comment suggestion > that would not work. Yes, that is tied to

Re: [Qemu-block] [Qemu-devel] [PATCH v4 0/2] block: Reject negative values for throttling options

On Tue, 01/19 10:50, Markus Armbruster wrote: > Fam Zheng writes: > > > v4: Add Max's rev-by in both patches, while fixing the "maxs" typo. > > > > v3: Address comments: > > - Add test for large value; [Berto] > > - Fix typos "negative" & "caught"; [Eric, Berto] > >

[Qemu-block] [PATCH v2 07/13] tests: Add test code for meta bitmap

Signed-off-by: Fam Zheng Reviewed-by: John Snow --- tests/test-hbitmap.c | 116 +++ 1 file changed, 116 insertions(+) diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index abcea0c..a341803 100644 ---

[Qemu-block] [PATCH v4 12/12] fdc: change auto fallback drive for ISA FDC to 288

The 2.88 drive is more suitable as a default because it can still read 1.44 images correctly, but the reverse is not true. Since there exist virtio-win drivers that are shipped on 2.88 floppy images, this patch will allow VMs booted without a floppy disk inserted to later insert a 2.88MB floppy

[Qemu-block] [PATCH v4 09/12] fdc: add physical disk sizes

2.88MB capable drives can accept 1.44MB floppies, for instance. To rework the pick_geometry function, we need to know if our current drive can even accept the type of disks we're considering. NB: This allows us to distinguish between all of the "total sectors" collisions between 1.20MB and 1.44MB

[Qemu-block] [PATCH v4 06/12] fdc: add pick_drive

Split apart pick_geometry by creating a pick_drive routine that will only ever called during device bring-up instead of relying on pick_geometry to be used in both cases. With this change, the drive field is changed to be 'write once'. It is not altered after the initialization routines exit.

[Qemu-block] [PATCH v4 10/12] fdc: rework pick_geometry

This one is the crazy one. fd_revalidate currently uses pick_geometry to tell if the diskette geometry has changed upon an eject/insert event, but it won't allow us to insert a 1.44MB diskette into a 2.88MB drive. This is inflexible. The new algorithm applies a new heuristic to guessing disk

[Qemu-block] [PATCH v4 07/12] fdc: Add fallback option

Currently, QEMU chooses a drive type automatically based on the inserted media. If there is no disk inserted, it chooses a 1.44MB drive type. Change this behavior to be configurable, but leave it defaulted to 1.44. This is not earnestly intended to be used by a user or a management library, but

[Qemu-block] [PATCH v2 00/13] Dirty bitmap changes for migration/persistence work

v2: Various changes addressing John's and Vladimir's comments: [02/13] typedefs: Add BdrvDirtyBitmap Skip HBitmapIter because we'll hide it soon. [John] [03/13] block: Move block dirty bitmap code to separate files [04/13] block: Remove unused typedef of

[Qemu-block] [PATCH v2 04/13] block: Remove unused typedef of BlockDriverDirtyHandler

Signed-off-by: Fam Zheng Reviewed-by: John Snow --- include/block/block.h | 2 -- 1 file changed, 2 deletions(-) diff --git a/include/block/block.h b/include/block/block.h index 339906b..054bf83 100644 --- a/include/block/block.h +++ b/include/block/block.h

[Qemu-block] [PATCH v4 11/12] qtest/fdc: Support for 2.88MB drives

The old test assumes a 1.44MB drive. Assert that the QEMU default drive is now either 1.44 or 2.88. Signed-off-by: John Snow --- tests/fdc-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/fdc-test.c b/tests/fdc-test.c index b5a4696..526d459

[Qemu-block] [PATCH v5 2/2] iotests: Test that throttle values ranges

Signed-off-by: Fam Zheng --- tests/qemu-iotests/051| 18 ++ tests/qemu-iotests/051.out| 39 +++ tests/qemu-iotests/051.pc.out | 39 +++ 3 files changed, 96 insertions(+) diff

[Qemu-block] [PATCH v5 0/2] block: Reject negative values for throttling options

v5: Add valid value testing in iotests. [Berto] Add details in commit message. [Markus] Use "%lld" in format string. [Markus] Fix ")" -> "]" in error message. [Kevin] Now the negative values are silently converted to a huge positive number because we are doing implicit casting from

[Qemu-block] [PATCH v5 1/2] blockdev: Error out on negative throttling option values

extract_common_blockdev_options() uses qemu_opt_get_number() to parse the bps/iops numbers to uint64_t, then converts to double and stores in ThrottleConfig. The actual parsing is done by strtoull() in parse_option_number(). Negative numbers are wrapped to large positive ones, and stored. We

Re: [Qemu-block] [Qemu-devel] [PATCH v5 0/2] block: Reject negative values for throttling options

On Wed, 01/20 08:25, Markus Armbruster wrote: > Fam Zheng writes: > > > v5: Add valid value testing in iotests. [Berto] > > Add details in commit message. [Markus] > > Use "%lld" in format string. [Markus] > > Fix ")" -> "]" in error message. [Kevin] > > > > Now the

[Qemu-block] [PATCH v3 11/13] nbd: enable use of TLS with NBD block driver

This modifies the NBD driver so that it is possible to request use of TLS. This is done by providing the 'tls-creds' parameter with the ID of a previously created QCryptoTLSCreds object. For example $QEMU -object tls-creds-x509,id=tls0,endpoint=client,\

[Qemu-block] [PATCH v3 08/13] nbd: allow setting of an export name for qemu-nbd server

The qemu-nbd server currently always uses the old style protocol since it never sets any export name. This is a problem because future TLS support will require use of the new style protocol negotiation. This adds a "--exportname NAME" argument to qemu-nbd which allows the user to set an explicit

[Qemu-block] [PATCH v3 01/13] nbd: convert block client to use I/O channels for connection setup

This converts the NBD block driver client to use the QIOChannelSocket class for initial connection setup. The NbdClientSession struct has two pointers, one to the master QIOChannelSocket providing the raw data channel, and one to a QIOChannel which is the current channel used for I/O. Initially

[Qemu-block] [PATCH v3 00/13] Implement TLS support to QEMU NBD server & client

This is an update of the series previously posted: v1: https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg06126.html v2: https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01580.html This series of patches implements support for TLS in the QEMU NBD server and client code. It is

[Qemu-block] [PATCH v3 02/13] nbd: convert qemu-nbd server to use I/O channels for connection setup

This converts the qemu-nbd server to use the QIOChannelSocket class for initial listener socket setup and accepting of client connections. Actual I/O is still being performed against the socket file descriptor using the POSIX socket APIs. In this initial conversion though, all I/O is still

[Qemu-block] [PATCH v3 03/13] nbd: convert blockdev NBD server to use I/O channels for connection setup

This converts the blockdev NBD server to use the QIOChannelSocket class for initial listener socket setup and accepting of client connections. Actual I/O is still being performed against the socket file descriptor using the POSIX socket APIs. Signed-off-by: Daniel P. Berrange

[Qemu-block] [PATCH v3 1/3] rbd: add support for getting password from QCryptoSecret object

Currently RBD passwords must be provided on the command line via $QEMU -drive file=rbd:pool/image:id=myname:\ key=QVFDVm41aE82SHpGQWhBQXEwTkN2OGp0SmNJY0UrSE9CbE1RMUE=:\ auth_supported=cephx This is insecure because the key is visible in the OS process listing.

[Qemu-block] [PATCH v3 12/13] nbd: enable use of TLS with qemu-nbd server

This modifies the qemu-nbd program so that it is possible to request the use of TLS with the server. It simply adds a new command line option --tls-creds which is used to provide the ID of a QCryptoTLSCreds object previously created via the --object command line option. For example qemu-nbd

[Qemu-block] [PATCH v3 04/13] nbd: convert to using I/O channels for actual socket I/O

Now that all callers are converted to use I/O channels for initial connection setup, it is possible to switch the core NBD protocol handling core over to use QIOChannel APIs for actual sockets I/O. Signed-off-by: Daniel P. Berrange --- block/nbd-client.c | 19 +++

[Qemu-block] [PATCH v3 09/13] nbd: pick first exported volume if no export name is requested

The NBD client is currently only capable of using the new style protocol negotiation if an explicit export name is provided. This is a problem, because TLS support will require use of the new style protocol in all cases, and we wish to keep the export name as an optional request for backwards

[Qemu-block] [PATCH v3 07/13] nbd: make client request fixed new style if advertized

If the server advertizes support for the fixed new style negotiation, the client should in turn enable new style. This will allow the client to negotiate further NBD options besides the export name. Signed-off-by: Daniel P. Berrange --- nbd/client.c | 27

[Qemu-block] [PATCH v3 05/13] nbd: invert client logic for negotiating protocol version

The nbd_receive_negotiate() method takes different code paths based on whether 'name == NULL', and then checks the expected protocol version in each branch. This patch inverts the logic, so that it takes different code paths based on what protocol version it receives and then checks if name is

[Qemu-block] [PATCH v3 3/3] iscsi: add support for getting CHAP password via QCryptoSecret API

The iSCSI driver currently accepts the CHAP password in plain text as a block driver property. This change adds a new "password-id" property that accepts the ID of a QCryptoSecret instance. $QEMU \ -object secret,id=sec0,filename=/home/berrange/example.pw \ -drive

Re: [Qemu-block] [PATCH v3 09/13] nbd: pick first exported volume if no export name is requested

On 19/01/2016 14:09, Daniel P. Berrange wrote: > The NBD client is currently only capable of using the new style > protocol negotiation if an explicit export name is provided. > This is a problem, because TLS support will require use of the > new style protocol in all cases, and we wish to keep

[Qemu-block] [PATCH v9 15/37] qom: Swap 'name' next to visitor in ObjectPropertyAccessor

Similar to the previous patch, it's nice to have all functions in the tree that involve a visitor and a name for conversion to or from QAPI to consistently stick the 'name' parameter next to the Visitor parameter. Done by manually changing include/qom/object.h and qom/object.c, then running this

[Qemu-block] [PATCH] qemu-iotests: Reduce racy output in 028

On my machine, './check -qcow2 028' was failing about 80% of the time, due to a race in how many times the repeated attempts to run 'info block-jobs' could occur before the job was done, showing up as a failure of fewer '(qemu) ' prompts than in the expected output. Silence the output during the

Re: [Qemu-block] [PATCH] block: Clean up includes

Am 18.01.2016 um 19:01 hat Peter Maydell geschrieben: > Clean up includes so that osdep.h is included first and headers > which it implies are not included manually. > > This commit was created with scripts/clean-includes. > > Signed-off-by: Peter Maydell Thanks,

Re: [Qemu-block] [PATCH v3 0/3] Use QCryptoSecret for block device passwords

On 19/01/2016 14:51, Daniel P. Berrange wrote: > This series was previously posted: > > v1: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html > v2: https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg03809.html > > The RBD, Curl and iSCSI block device drivers all

Re: [Qemu-block] [PATCH v3 0/3] Use QCryptoSecret for block device passwords

On Tue, Jan 19, 2016 at 05:32:35PM +0100, Paolo Bonzini wrote: > > > On 19/01/2016 14:51, Daniel P. Berrange wrote: > > This series was previously posted: > > > > v1: https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg04365.html > > v2:

Re: [Qemu-block] [PATCH v2 0/9] block/qcow2: Migration handoff fixes and cleanups

Am 13.01.2016 um 17:37 hat Kevin Wolf geschrieben: > This is the non-controversal part of the qcow2 locking series, which just > makes > the state of images with respect to live migration clearer and fixes a few > bugs > we have in the handoff procedure and with writing the qcow2 feature table.

Re: [Qemu-block] [PATCH v3 04/13] nbd: convert to using I/O channels for actual socket I/O

On Tue, Jan 19, 2016 at 05:08:12PM +0100, Paolo Bonzini wrote: > > > On 19/01/2016 14:09, Daniel P. Berrange wrote: > > + * This will update @iov so that its head is advanced > > + * by @skip bytes. To do this, zero or more complete > > + * elements of @iov will be skipped over. The new head > >

Re: [Qemu-block] [PATCH] qemu-iotests: Reduce racy output in 028

Am 18.12.2015 um 22:49 hat John Snow geschrieben: > > > On 12/18/2015 03:46 PM, Eric Blake wrote: > > On 12/11/2015 06:23 PM, John Snow wrote: > >> On 12/10/2015 10:27 PM, Eric Blake wrote: > >>> On my machine, './check -qcow2 028' was failing about 80% of the > >>> time, due to a race in how

Re: [Qemu-block] [PATCH v3 08/13] nbd: allow setting of an export name for qemu-nbd server

On 19/01/2016 14:09, Daniel P. Berrange wrote: > The qemu-nbd server currently always uses the old style protocol > since it never sets any export name. This is a problem because > future TLS support will require use of the new style protocol > negotiation. > > This adds a "--exportname NAME"

Re: [Qemu-block] [PATCH v3 09/13] nbd: pick first exported volume if no export name is requested

On Tue, Jan 19, 2016 at 05:14:32PM +0100, Paolo Bonzini wrote: > > > On 19/01/2016 14:09, Daniel P. Berrange wrote: > > The NBD client is currently only capable of using the new style > > protocol negotiation if an explicit export name is provided. > > This is a problem, because TLS support will

Re: [Qemu-block] [PATCH v3 08/13] nbd: allow setting of an export name for qemu-nbd server

On Tue, Jan 19, 2016 at 05:12:01PM +0100, Paolo Bonzini wrote: > > > On 19/01/2016 14:09, Daniel P. Berrange wrote: > > The qemu-nbd server currently always uses the old style protocol > > since it never sets any export name. This is a problem because > > future TLS support will require use of

Re: [Qemu-block] [PATCH 1/5] ide: Prohibit RESET on IDE drives

On 01/19/2016 06:48 AM, Paolo Bonzini wrote: > > > On 19/01/2016 05:51, John Snow wrote: >> +/* Only RESET is allowed to an ATAPI device while BSY and/or DRQ are >> set. */ >> +if (s->status & (BUSY_STAT|DRQ_STAT)) { >> +if (!(val == WIN_DEVICE_RESET) && (s->drive_kind ==

[Qemu-block] [PATCH v2 4/6] ide: replace blk_drain_all by blk_drain

Target the drain for just one device. Signed-off-by: John Snow --- hw/ide/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index 7d32266..cf0b5ec 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -648,7 +648,7 @@ void

[Qemu-block] [PATCH v2 5/6] ide: Add silent DRQ cancellation

Split apart the ide_transfer_stop function into two versions: one that interrupts and one that doesn't. The one that doesn't can be used to halt any PIO transfers that are in the DRQ phase. It will not halt any PIO transfers that are currently in the process of buffering data for the guest to

[Qemu-block] [PATCH v2 1/6] ide: Prohibit RESET on IDE drives

This command is meant for ATAPI devices only, prohibit acknowledging it with a command aborted response when an IDE device is busy. Signed-off-by: John Snow --- hw/ide/core.c | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/hw/ide/core.c

[Qemu-block] [PATCH v2 2/6] ide: code motion

Shuffle the reset function upwards. Signed-off-by: John Snow --- hw/ide/core.c | 116 +- 1 file changed, 58 insertions(+), 58 deletions(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index d121f3d..10f2702 100644 ---

[Qemu-block] [PATCH v2 0/6] ide: fix atapi software reset

The ATAPI software reset function is implemented somewhat lackadaisically. Firstly, it is valid only for ATAPI drives - not HDs. If a HD should receive this command while BSY, it should be ignored like any other command instead of aborted. A non-BSY HD is free to abort the command in the usual