[Qemu-block] [RFC v4 16/21] blockjobs: add waiting status

2018-02-23 Thread John Snow
For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their

[Qemu-block] [RFC v4 02/21] blockjobs: model single jobs as transactions

2018-02-23 Thread John Snow
model all independent jobs as single job transactions. It's one less case we have to worry about when we add more states to the transition machine. This way, we can just treat all job lifetimes exactly the same. This helps tighten assertions of the STM graph and removes some conditionals that

[Qemu-block] [RFC v4 14/21] blockjobs: add block_job_txn_apply function

2018-02-23 Thread John Snow
Simply apply a function transaction-wide. A few more uses of this in forthcoming patches. Signed-off-by: John Snow --- blockjob.c | 24 +++- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/blockjob.c b/blockjob.c index 431ce9c220..8f02c03880

Re: [Qemu-block] [Qemu-devel] [RFC v4 00/21] blockjobs: add explicit job management

2018-02-23 Thread no-reply
Hi, This series failed build test on ppcbe host. Please find the details below. Type: series Message-id: 20180223235142.21501-1-js...@redhat.com Subject: [Qemu-devel] [RFC v4 00/21] blockjobs: add explicit job management === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked

[Qemu-block] [RFC v4 11/21] blockjobs: add block_job_dismiss

2018-02-23 Thread John Snow
For jobs that have reached their CONCLUDED state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via block-job-dismiss. This gives management APIs the

[Qemu-block] [RFC v4 09/21] blockjobs: add CONCLUDED state

2018-02-23 Thread John Snow
add a new state "CONCLUDED" that identifies a job that has ceased all operations. The wording was chosen to avoid any phrasing that might imply success, error, or cancellation. The task has simply ceased all operation and can never again perform any work. ("finished", "done", and "completed"

[Qemu-block] [RFC v4 10/21] blockjobs: add NULL state

2018-02-23 Thread John Snow
Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Transitions: Created -> Null: Early failure event before

[Qemu-block] [RFC v4 18/21] blockjobs: add block-job-finalize

2018-02-23 Thread John Snow
Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if manual completion mode has been requested. This allows us to perform graph changes in prepare and/or commit so that graph

[Qemu-block] [RFC v4 07/21] blockjobs: add block_job_verb permission table

2018-02-23 Thread John Snow
Which commands ("verbs") are appropriate for jobs in which state is also somewhat burdensome to keep track of. As of this commit, it looks rather useless, but begins to look more interesting the more states we add to the STM table. A recurring theme is that no verb will apply to an 'undefined'

[Qemu-block] [RFC v4 05/21] blockjobs: add state transition table

2018-02-23 Thread John Snow
The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. Transitions: Undefined -> Created: During job initialization.

[Qemu-block] [RFC v4 20/21] iotests: test manual job dismissal

2018-02-23 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/056 | 195 + tests/qemu-iotests/056.out | 4 +- 2 files changed, 197 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056 index

[Qemu-block] [RFC v4 21/21] blockjobs: add manual_mgmt option to transactions

2018-02-23 Thread John Snow
This allows us to easily force the option for all jobs belonging to a transaction to ensure consistency with how all those jobs will be handled. This is purely a convenience. Signed-off-by: John Snow --- blockdev.c| 7 ++- blockjob.c| 10

[Qemu-block] [RFC v4 01/21] blockjobs: fix set-speed kick

2018-02-23 Thread John Snow
If speed is '0' it's not actually "less than" the previous speed. Kick the job in this case too. Signed-off-by: John Snow --- blockjob.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/blockjob.c b/blockjob.c index 3f52f29f75..24833ef30f 100644 ---

[Qemu-block] [RFC v4 00/21] blockjobs: add explicit job management

2018-02-23 Thread John Snow
This series seeks to address two distinct but closely related issues concerning the job management API. (1) For jobs that complete when a monitor is not attached and receiving events or notifications, there's no way to discern the job's final return code. Jobs must remain in the query

Re: [Qemu-block] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()

2018-02-23 Thread Eric Blake
On 02/23/2018 11:05 AM, Kevin Wolf wrote: Am 23.02.2018 um 17:43 hat Eric Blake geschrieben: OFFSET_VALID | DATA might be excusable because I can see that it's convenient that a protocol driver refers to itself as *file instead of returning NULL there and then the offset is valid (though it

[Qemu-block] [RFC v4 08/21] blockjobs: add ABORTING state

2018-02-23 Thread John Snow
Add a new state ABORTING. This makes transitions from normative states to error states explicit in the STM, and serves as a disambiguation for which states may complete normally when normal end-states (CONCLUDED) are added in future commits. Notably, Paused/Standby jobs do not transition

[Qemu-block] [RFC v4 19/21] blockjobs: Expose manual property

2018-02-23 Thread John Snow
Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow --- blockdev.c | 19 ---

[Qemu-block] [RFC v4 15/21] blockjobs: add prepare callback

2018-02-23 Thread John Snow
Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have fairly

[Qemu-block] [RFC v4 03/21] blockjobs: add manual property

2018-02-23 Thread John Snow
This property will be used to opt-in to the new BlockJobs workflow that allows a tighter, more explicit control over transitions from one runstate to another. While we're here, fix up the documentation for block_job_create a little bit. Signed-off-by: John Snow --- blockjob.c

[Qemu-block] [RFC v4 17/21] blockjobs: add PENDING status and event

2018-02-23 Thread John Snow
For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting," we add a new "pending" event. For

[Qemu-block] [RFC v4 06/21] iotests: add pause_wait

2018-02-23 Thread John Snow
Split out the pause command into the actual pause and the wait. Not every usage presently needs to resubmit a pause request. The intent with the next commit will be to explicitly disallow redundant or meaningless pause/resume requests, so the tests need to become more judicious to reflect that.

[Qemu-block] [RFC v4 13/21] blockjobs: add commit, abort, clean helpers

2018-02-23 Thread John Snow
The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow --- blockjob.c | 35 ++- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git

[Qemu-block] [RFC v4 12/21] blockjobs: ensure abort is called for cancelled jobs

2018-02-23 Thread John Snow
Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's implementation of .commit checks for

[Qemu-block] [RFC v4 04/21] blockjobs: add status enum

2018-02-23 Thread John Snow
We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. It would help to have a more explicit bookkeeping of the state of blockjobs. To this end, add a new "status" field and add our existing states in a redundant manner alongside the bools they

Re: [Qemu-block] [Qemu-devel] [PATCH] scsi: Remove automatic creation of SCSI controllers with -drive if=scsi

2018-02-23 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. N/A. Internal error while reading log file --- Email generated automatically by Patchew [http://patchew.org/]. Please send your feedback to patchew-de...@freelists.org

Re: [Qemu-block] [Qemu-devel] [PATCH 0/9] nbd block status base:allocation

2018-02-23 Thread no-reply
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 1518702707-7077-1-git-send-email-vsement...@virtuozzo.com Subject: [Qemu-devel] [PATCH

Re: [Qemu-block] [PATCH v2 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

2018-02-23 Thread Kevin Wolf
Am 23.02.2018 um 17:43 hat Max Reitz geschrieben: > On 2018-02-23 17:19, Kevin Wolf wrote: > > Am 23.02.2018 um 00:25 hat Max Reitz geschrieben: > >> On 2018-02-21 14:53, Kevin Wolf wrote: > >>> With the conversion to a QAPI options object, the function is now > >>> prepared to be used in a

Re: [Qemu-block] [PATCH v2 19/36] rbd: Factor out qemu_rbd_connect()

2018-02-23 Thread Kevin Wolf
Am 23.02.2018 um 00:10 hat Max Reitz geschrieben: > On 2018-02-21 14:53, Kevin Wolf wrote: > > The code to establish an RBD connection is duplicated between open and > > create. In order to be able to share the code, factor out the code from > > qemu_rbd_open() as a first step. > > > >

Re: [Qemu-block] [PATCH v2 27/36] sheepdog: QAPIfy "redundacy" create option

2018-02-23 Thread Kevin Wolf
Am 21.02.2018 um 14:53 hat Kevin Wolf geschrieben: > The "redundacy" option for Sheepdog image creation is currently a string > that can encode one or two integers depending on its format, which at > the same time implicitly selects a mode. > > This patch turns it into a QAPI union and converts

Re: [Qemu-block] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()

2018-02-23 Thread Kevin Wolf
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben: > > OFFSET_VALID | DATA might be excusable because I can see that it's > > convenient that a protocol driver refers to itself as *file instead of > > returning NULL there and then the offset is valid (though it would be > > pointless to actually

Re: [Qemu-block] [PATCH] vl: introduce vm_shutdown()

2018-02-23 Thread Fam Zheng
On Tue, 02/20 13:10, Stefan Hajnoczi wrote: > 1. virtio_scsi_handle_cmd_vq() racing with iothread_stop_all() hits the >virtio_scsi_ctx_check() assertion failure because the BDS AioContext >has been modified by iothread_stop_all(). Does this patch fix the issue completely? IIUC

[Qemu-block] [PATCH 5/5] ide: introduce ide_transfer_start_norecurse

2018-02-23 Thread Paolo Bonzini
For the case where the end_transfer_func is also the caller of ide_transfer_start, the mutual recursion can lead to unlimited stack usage. Introduce a new version that can be used to change tail recursion into a loop, and use it in trace_ide_atapi_cmd_reply_end. Signed-off-by: Paolo Bonzini

[Qemu-block] [PATCH 4/5] atapi: call ide_set_irq before ide_transfer_start

2018-02-23 Thread Paolo Bonzini
The ATAPI_INT_REASON_IO interrupt is raised when I/O starts, but in the AHCI case ide_set_irq was actually called at the end of a mutual recursion. Move it early, with the side effect that ide_transfer_start becomes a tail call in ide_atapi_cmd_reply_end. Signed-off-by: Paolo Bonzini

Re: [Qemu-block] [PATCH v8 09/21] null: Switch to .bdrv_co_block_status()

2018-02-23 Thread Eric Blake
On 02/14/2018 06:05 AM, Kevin Wolf wrote: +static int coroutine_fn null_co_block_status(BlockDriverState *bs, if (s->read_zeroes) { -return BDRV_BLOCK_OFFSET_VALID | start | BDRV_BLOCK_ZERO; -} else { -return BDRV_BLOCK_OFFSET_VALID | start; +ret |=

Re: [Qemu-block] [PATCH v2 17/36] gluster: Support .bdrv_co_create

2018-02-23 Thread Eric Blake
On 02/21/2018 07:53 AM, Kevin Wolf wrote: This adds the .bdrv_co_create driver callback to gluster, which enables image creation over QMP. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 18 ++- block/gluster.c | 135

Re: [Qemu-block] [PATCH v2 18/36] rbd: Fix use after free in qemu_rbd_set_keypairs() error path

2018-02-23 Thread Eric Blake
On 02/21/2018 07:53 AM, Kevin Wolf wrote: If we want to include the invalid option name in the error message, we can't free the string earlier than that. Signed-off-by: Kevin Wolf --- block/rbd.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) D'oh. Should this

Re: [Qemu-block] [PATCH v2 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

2018-02-23 Thread Kevin Wolf
Am 23.02.2018 um 00:25 hat Max Reitz geschrieben: > On 2018-02-21 14:53, Kevin Wolf wrote: > > With the conversion to a QAPI options object, the function is now > > prepared to be used in a .bdrv_co_create implementation. > > > > Signed-off-by: Kevin Wolf > > -*s_snap =

Re: [Qemu-block] [PATCH v2 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

2018-02-23 Thread Max Reitz
On 2018-02-23 17:19, Kevin Wolf wrote: > Am 23.02.2018 um 00:25 hat Max Reitz geschrieben: >> On 2018-02-21 14:53, Kevin Wolf wrote: >>> With the conversion to a QAPI options object, the function is now >>> prepared to be used in a .bdrv_co_create implementation. >>> >>> Signed-off-by: Kevin Wolf

[Qemu-block] [PATCH 2/5] ide: push end_transfer callback to ide_transfer_halt

2018-02-23 Thread Paolo Bonzini
The callback must be invoked once we get out of the DRQ phase; because all end_transfer_funcs end up invoking ide_transfer_stop, call it there. While at it, remove the "notify" argument from ide_transfer_halt; the code can simply be moved to ide_transfer_stop. Old PATA controllers have no

[Qemu-block] [PATCH 3/5] ide: do not set s->end_transfer_func to ide_transfer_cancel

2018-02-23 Thread Paolo Bonzini
There is code checking s->end_transfer_func and it was not taught about ide_transfer_cancel. We can just use ide_transfer_stop because s->end_transfer_func is only ever called in the DRQ phase: after ide_transfer_cancel, the value of s->end_transfer_func is only used as a marker and never used to

[Qemu-block] [RFC PATCH 0/5] atapi: change unlimited recursion to while loop

2018-02-23 Thread Paolo Bonzini
Real hardware doesn't have an unlimited stack, so the unlimited recursion in the ATAPI code smells a bit. In fact, the call to ide_transfer_start easily becomes a tail call with a small change to the code (patch 4). The remaining four patches move code around so as to the turn the call back to

[Qemu-block] [PATCH 1/5] ide: push call to end_transfer_func out of start_transfer callback

2018-02-23 Thread Paolo Bonzini
Split the PIO transfer across two callbacks, thus pushing the (possibly recursive) call to end_transfer_func up one level and out of the AHCI-specific code. Signed-off-by: Paolo Bonzini --- hw/ide/ahci.c | 7 ++- hw/ide/core.c | 9 ++---

Re: [Qemu-block] [PATCH v2 33/36] file-posix: Fix no-op bdrv_truncate() with falloc preallocation

2018-02-23 Thread Eric Blake
On 02/21/2018 07:54 AM, Kevin Wolf wrote: If bdrv_truncate() is called, but the requested size is the same as before, don't call posix_fallocate(), which returns -EINVAL for length zero and would therefore make bdrv_truncate() fail. The problem can be triggered by creating a zero-sized raw

Re: [Qemu-block] [PATCH v2 18/36] rbd: Fix use after free in qemu_rbd_set_keypairs() error path

2018-02-23 Thread Kevin Wolf
Am 23.02.2018 um 16:15 hat Eric Blake geschrieben: > On 02/21/2018 07:53 AM, Kevin Wolf wrote: > > If we want to include the invalid option name in the error message, we > > can't free the string earlier than that. > > > > Signed-off-by: Kevin Wolf > > --- > > block/rbd.c | 3

[Qemu-block] [PATCH v3 04/36] qcow2: Pass BlockdevCreateOptions to qcow2_create2()

2018-02-23 Thread Kevin Wolf
All of the simple options are now passed to qcow2_create2() in a BlockdevCreateOptions object. Still missing: node-name and the encryption options. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/qcow2.c | 190

[Qemu-block] [PATCH v3 05/36] qcow2: Use BlockdevRef in qcow2_create2()

2018-02-23 Thread Kevin Wolf
Instead of passing a separate BlockDriverState* into qcow2_create2(), make use of the BlockdevRef that is included in BlockdevCreateOptions. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz ---

[Qemu-block] [PATCH v3 07/36] qcow2: Handle full/falloc preallocation in qcow2_create2()

2018-02-23 Thread Kevin Wolf
Once qcow2_create2() can be called directly on an already existing node, we must provide the 'full' and 'falloc' preallocation modes outside of creating the image on the protocol layer. Fortunately, we have preallocated truncate now which can provide this functionality. Signed-off-by: Kevin Wolf

[Qemu-block] [PATCH v3 00/36] x-blockdev-create for protocols and qcow2

2018-02-23 Thread Kevin Wolf
This series implements a minimal QMP command that allows to create an image file on the protocol level or an image format on a given block node. Eventually, the interface is going to change to some kind of an async command (possibly a (non-)block job), but that will require more work on the job

[Qemu-block] [PATCH v3 01/36] block/qapi: Introduce BlockdevCreateOptions

2018-02-23 Thread Kevin Wolf
This creates a BlockdevCreateOptions union type that will contain all of the options for image creation. We'll start out with an empty struct type BlockdevCreateNotSupported for all drivers. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max

[Qemu-block] [PATCH v3 06/36] qcow2: Use QCryptoBlockCreateOptions in qcow2_create2()

2018-02-23 Thread Kevin Wolf
Instead of passing the encryption format name and the QemuOpts down, use the QCryptoBlockCreateOptions contained in BlockdevCreateOptions. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- block/qcow2.c | 62

[Qemu-block] [PATCH v3 03/36] qcow2: Let qcow2_create() handle protocol layer

2018-02-23 Thread Kevin Wolf
Currently, qcow2_create() only parses the QemuOpts and then calls qcow2_create2() for the actual image creation, which includes both the creation of the actual file on the file system and writing a valid empty qcow2 image into that file. The plan is that qcow2_create2() becomes the function that

[Qemu-block] [PATCH v3 02/36] block/qapi: Add qcow2 create options to schema

2018-02-23 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- qapi/block-core.json | 45 - 1 file changed, 44 insertions(+), 1 deletion(-) diff --git a/qapi/block-core.json

[Qemu-block] [PATCH v3 11/36] qdict: Introduce qdict_rename_keys()

2018-02-23 Thread Kevin Wolf
A few block drivers will need to rename .bdrv_create options for their QAPIfication, so let's have a helper function for that. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- include/qapi/qmp/qdict.h | 6

[Qemu-block] [PATCH v3 08/36] util: Add qemu_opts_to_qdict_filtered()

2018-02-23 Thread Kevin Wolf
This allows, given a QemuOpts for a QemuOptsList that was merged from multiple QemuOptsList, to only consider those options that exist in one specific list. Block drivers need this to separate format-layer create options from protocol-level options. Signed-off-by: Kevin Wolf

[Qemu-block] [PATCH v3 15/36] file-posix: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to file, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- qapi/block-core.json | 20 +- block/file-posix.c |

[Qemu-block] [PATCH v3 23/36] rbd: Assign s->snap/image_name in qemu_rbd_open()

2018-02-23 Thread Kevin Wolf
Now that the options are already available in qemu_rbd_open() and not only parsed in qemu_rbd_connect(), we can assign s->snap and s->image_name there instead of passing the fields by reference to qemu_rbd_connect(). Signed-off-by: Kevin Wolf --- block/rbd.c | 14

[Qemu-block] [PATCH v3 24/36] rbd: Use qemu_rbd_connect() in qemu_rbd_do_create()

2018-02-23 Thread Kevin Wolf
This is almost exactly the same code. The differences are that qemu_rbd_connect() supports BlockdevOptionsRbd.server and that the cache mode is set explicitly. Supporting 'server' is a welcome new feature for image creation. Caching is disabled by default, so leave it that way. Signed-off-by:

[Qemu-block] [PATCH v3 10/36] test-qemu-opts: Test qemu_opts_to_qdict_filtered()

2018-02-23 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- tests/test-qemu-opts.c | 125 + 1 file changed, 125 insertions(+) diff --git a/tests/test-qemu-opts.c

[Qemu-block] [PATCH v3 16/36] file-win32: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to file-win32, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- block/file-win32.c | 45

[Qemu-block] [PATCH v3 25/36] nfs: Use QAPI options in nfs_client_open()

2018-02-23 Thread Kevin Wolf
Using the QAPI visitor to turn all options into QAPI BlockdevOptionsNfs simplifies the code a lot. It will also be useful for implementing the QAPI based .bdrv_co_create callback. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/nfs.c | 176

[Qemu-block] [PATCH v3 27/36] sheepdog: QAPIfy "redundancy" create option

2018-02-23 Thread Kevin Wolf
The "redundancy" option for Sheepdog image creation is currently a string that can encode one or two integers depending on its format, which at the same time implicitly selects a mode. This patch turns it into a QAPI union and converts the string into such a QAPI object before interpreting the

[Qemu-block] [PATCH v3 29/36] ssh: Use QAPI BlockdevOptionsSsh object

2018-02-23 Thread Kevin Wolf
Create a BlockdevOptionsSsh object in connect_to_ssh() and take the options from there. 'host_key_check' is still processed separately because it's not in the schema yet. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/ssh.c | 136

[Qemu-block] [PATCH v3 34/36] block: Fail bdrv_truncate() with negative size

2018-02-23 Thread Kevin Wolf
Most callers have their own checks, but something like this should also be checked centrally. As it happens, x-blockdev-create can pass negative image sizes to format drivers (because there is no QAPI type that would reject negative numbers) and triggers the check added by this patch.

[Qemu-block] [PATCH v3 32/36] ssh: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to ssh, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 16 - block/ssh.c | 92

[Qemu-block] [PATCH v3 31/36] ssh: Pass BlockdevOptionsSsh to connect_to_ssh()

2018-02-23 Thread Kevin Wolf
Move the parsing of the QDict options up to the callers, in preparation for the .bdrv_co_create implementation that directly gets a QAPI type. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/ssh.c | 34 +- 1 file

[Qemu-block] [PATCH v3 33/36] file-posix: Fix no-op bdrv_truncate() with falloc preallocation

2018-02-23 Thread Kevin Wolf
If bdrv_truncate() is called, but the requested size is the same as before, don't call posix_fallocate(), which returns -EINVAL for length zero and would therefore make bdrv_truncate() fail. The problem can be triggered by creating a zero-sized raw image with 'falloc' preallocation mode.

[Qemu-block] [PATCH v3 36/36] qemu-iotests: Test ssh image creation over QMP

2018-02-23 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- tests/qemu-iotests/207 | 261 + tests/qemu-iotests/207.out | 75 + tests/qemu-iotests/group | 1 + 3 files changed, 337 insertions(+)

[Qemu-block] [PATCH v3 12/36] qcow2: Use visitor for options in qcow2_create()

2018-02-23 Thread Kevin Wolf
Instead of manually creating the BlockdevCreateOptions object, use a visitor to parse the given options into the QAPI object. This involves translation from the old command line syntax to the syntax mandated by the QAPI schema. Option names are still checked against qcow2_create_opts, so only the

[Qemu-block] [PATCH v3 09/36] test-qemu-opts: Test qemu_opts_append()

2018-02-23 Thread Kevin Wolf
Basic test for merging two QemuOptsLists. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake Reviewed-by: Max Reitz --- tests/test-qemu-opts.c | 128 + 1 file changed, 128 insertions(+)

[Qemu-block] [PATCH v3 14/36] block: x-blockdev-create QMP command

2018-02-23 Thread Kevin Wolf
This adds a synchronous x-blockdev-create QMP command that can create qcow2 images on a given node name. We don't want to block while creating an image, so this is not the final interface in all aspects, but BlockdevCreateOptionsQcow2 and .bdrv_co_create() are what they actually might look like

[Qemu-block] [PATCH v3 13/36] block: Make bdrv_is_whitelisted() public

2018-02-23 Thread Kevin Wolf
We'll use a separate source file for image creation, and we need to check there whether the requested driver is whitelisted. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- include/block/block.h | 1 +

[Qemu-block] [PATCH v3 17/36] gluster: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to gluster, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- qapi/block-core.json | 18 ++- block/gluster.c | 135

[Qemu-block] [PATCH v3 30/36] ssh: QAPIfy host-key-check option

2018-02-23 Thread Kevin Wolf
This makes the host-key-check option available in blockdev-add. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 63 +++-- block/ssh.c | 88

[Qemu-block] [PATCH v3 21/36] rbd: Pass BlockdevOptionsRbd to qemu_rbd_connect()

2018-02-23 Thread Kevin Wolf
With the conversion to a QAPI options object, the function is now prepared to be used in a .bdrv_co_create implementation. Signed-off-by: Kevin Wolf --- block/rbd.c | 109 +--- 1 file changed, 53 insertions(+), 56

[Qemu-block] [PATCH v3 20/36] rbd: Remove non-schema options from runtime_opts

2018-02-23 Thread Kevin Wolf
Instead of the QemuOpts in qemu_rbd_connect(), we want to use QAPI objects. As a preparation, fetch those options directly from the QDict that .bdrv_open() supports in the rbd driver and that are not in the schema. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz

[Qemu-block] [PATCH v3 18/36] rbd: Fix use after free in qemu_rbd_set_keypairs() error path

2018-02-23 Thread Kevin Wolf
If we want to include the invalid option name in the error message, we can't free the string earlier than that. Cc: qemu-sta...@nongnu.org Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: Eric Blake --- block/rbd.c | 3 ++-

[Qemu-block] [PATCH v3 19/36] rbd: Factor out qemu_rbd_connect()

2018-02-23 Thread Kevin Wolf
The code to establish an RBD connection is duplicated between open and create. In order to be able to share the code, factor out the code from qemu_rbd_open() as a first step. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- block/rbd.c | 100

[Qemu-block] [PATCH v3 22/36] rbd: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to rbd, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 19 ++- block/rbd.c | 146

[Qemu-block] [PATCH v3 26/36] nfs: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to nfs, which enables image creation over QMP. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- qapi/block-core.json | 16 +++- block/nfs.c | 74

[Qemu-block] [PATCH v3 28/36] sheepdog: Support .bdrv_co_create

2018-02-23 Thread Kevin Wolf
This adds the .bdrv_co_create driver callback to sheepdog, which enables image creation over QMP. Signed-off-by: Kevin Wolf --- qapi/block-core.json | 24 - block/sheepdog.c | 242 +++ 2 files changed, 191 insertions(+),

[Qemu-block] [PATCH v3 35/36] qemu-iotests: Test qcow2 over file image creation with QMP

2018-02-23 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- tests/qemu-iotests/206 | 436 + tests/qemu-iotests/206.out | 209 ++ tests/qemu-iotests/group | 1 + 3 files changed, 646

Re: [Qemu-block] [Qemu-devel] [PATCH v2] block: Fix qemu crash when using scsi-block

2018-02-23 Thread Deepa Srinivasan
Stefan, Kevin - Ping, to take this patch. Thanks. On 01/29/2018 07:51 AM, Stefan Hajnoczi wrote: On Fri, Dec 15, 2017 at 04:59:13PM -0800, Deepa Srinivasan wrote: Starting qemu with the following arguments causes qemu to segfault: ... -device lsi,id=lsi0 -drive

Re: [Qemu-block] Limiting coroutine stack usage

2018-02-23 Thread Paolo Bonzini
On 22/02/2018 18:06, John Snow wrote: > > > On 02/22/2018 05:57 AM, Kevin Wolf wrote: >> Am 20.02.2018 um 22:54 hat Paolo Bonzini geschrieben: >>> On 20/02/2018 18:04, Peter Lieven wrote: Hi, I remember we discussed a long time ago to limit the stack usage of all functions

Re: [Qemu-block] [PATCH] iotests: Test abnormally large size in compressed cluster descriptor

2018-02-23 Thread Alberto Garcia
On Thu 22 Feb 2018 08:00:08 PM CET, Eric Blake wrote: >> One consequence of this is that even if the size field is larger than >> it needs to be QEMU can handle it just fine: it will read more data >> from disk but it will ignore the extra bytes. > > (is that true even for the corner case when the

Re: [Qemu-block] [PATCH] qemu-img: Make resize error message more general

2018-02-23 Thread Max Reitz
On 2018-02-05 17:27, Max Reitz wrote: > The issue: > > $ qemu-img resize -f qcow2 foo.qcow2 > qemu-img: Expecting one image file name > Try 'qemu-img --help' for more information > > So we gave an image file name, but we omitted the length. qemu-img > thinks the last argument is always

Re: [Qemu-block] [PATCH 0/3] block/ssh: Add basic .bdrv_truncate()

2018-02-23 Thread Max Reitz
On 2018-02-14 21:49, Max Reitz wrote: > For (x-)blockdev-create, all protocol drivers that support image > creation also need to offer a .bdrv_truncate() implementation that > matches in features. A previous series of mine brought gluster's and > sheepdog's implementation up to par regarding

Re: [Qemu-block] [PATCH v3] qcow2: Replace align_offset() with ROUND_UP()

2018-02-23 Thread Max Reitz
On 2018-02-15 14:10, Alberto Garcia wrote: > The align_offset() function is equivalent to the ROUND_UP() macro so > there's no need to use the former. The ROUND_UP() name is also a bit > more explicit. > > This patch uses ROUND_UP() instead of the slower QEMU_ALIGN_UP() > because align_offset()

Re: [Qemu-block] [Qemu-devel] [PATCH 0/9] nbd block status base:allocation

2018-02-23 Thread no-reply
Hi, This series failed build test on ppcle host. Please find the details below. Subject: [Qemu-devel] [PATCH 0/9] nbd block status base:allocation Type: series Message-id: 1518702707-7077-1-git-send-email-vsement...@virtuozzo.com === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be

[Qemu-block] Intermittent failure of iotest 203

2018-02-23 Thread Max Reitz
Hi, iotest 203 relatively often fails for me, at least when run in parallel. When I run the following concurrently on four shells: $ while TEST_DIR=/tmp/t1 ./check -T -qcow2 203; do; done $ while TEST_DIR=/tmp/t2 ./check -T -qcow2 203; do; done $ while TEST_DIR=/tmp/t3 ./check -T -qcow2 203;

Re: [Qemu-block] [PATCH v2 1/2] iotest 033: add misaligned write-zeroes test via truncate

2018-02-23 Thread Max Reitz
On 2018-02-12 14:14, Anton Nefedov wrote: > This new test case only makes sense for qcow2 while iotest 033 is generic; > however it matches the test purpose perfectly and also 033 contains those > do_test() tricks to pass the alignment, which won't look nice being > duplicated in other tests or

[Qemu-block] [PATCH v2] iotests: Test abnormally large size in compressed cluster descriptor

2018-02-23 Thread Alberto Garcia
L2 entries for compressed clusters have a field that indicates the number of sectors used to store the data in the image. That's however not the size of the compressed data itself, just the number of sectors where that data is located. The actual data size is usually not a multiple of the sector

Re: [Qemu-block] [PATCH v2] iotests: Test abnormally large size in compressed cluster descriptor

2018-02-23 Thread Eric Blake
On 02/23/2018 06:50 AM, Alberto Garcia wrote: L2 entries for compressed clusters have a field that indicates the number of sectors used to store the data in the image. That's however not the size of the compressed data itself, just the number of sectors where that data is located. The actual