For jobs that are stuck waiting on others in a transaction, it would
be nice to know that they are no longer "running" in that sense, but
instead are waiting on other jobs in the transaction.
Jobs that are "waiting" in this sense cannot be meaningfully altered
any longer as they have left their
model all independent jobs as single job transactions.
It's one less case we have to worry about when we add more states to the
transition machine. This way, we can just treat all job lifetimes exactly
the same. This helps tighten assertions of the STM graph and removes some
conditionals that
Simply apply a function transaction-wide.
A few more uses of this in forthcoming patches.
Signed-off-by: John Snow
---
blockjob.c | 24 +++-
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/blockjob.c b/blockjob.c
index 431ce9c220..8f02c03880
Hi,
This series failed build test on ppcbe host. Please find the details below.
Type: series
Message-id: 20180223235142.21501-1-js...@redhat.com
Subject: [Qemu-devel] [RFC v4 00/21] blockjobs: add explicit job management
=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be invoked
For jobs that have reached their CONCLUDED state, prior to having their
last reference put down (meaning jobs that have completed successfully,
unsuccessfully, or have been canceled), allow the user to dismiss the
job's lingering status report via block-job-dismiss.
This gives management APIs the
add a new state "CONCLUDED" that identifies a job that has ceased all
operations. The wording was chosen to avoid any phrasing that might
imply success, error, or cancellation. The task has simply ceased all
operation and can never again perform any work.
("finished", "done", and "completed"
Add a new state that specifically demarcates when we begin to permanently
demolish a job after it has performed all work. This makes the transition
explicit in the STM table and highlights conditions under which a job may
be demolished.
Transitions:
Created -> Null: Early failure event before
Instead of automatically transitioning from PENDING to CONCLUDED, gate
the .prepare() and .commit() phases behind an explicit acknowledgement
provided by the QMP monitor if manual completion mode has been requested.
This allows us to perform graph changes in prepare and/or commit so that
graph
Which commands ("verbs") are appropriate for jobs in which state is
also somewhat burdensome to keep track of.
As of this commit, it looks rather useless, but begins to look more
interesting the more states we add to the STM table.
A recurring theme is that no verb will apply to an 'undefined'
The state transition table has mostly been implied. We're about to make
it a bit more complex, so let's make the STM explicit instead.
Perform state transitions with a function that for now just asserts the
transition is appropriate.
Transitions:
Undefined -> Created: During job initialization.
Signed-off-by: John Snow
---
tests/qemu-iotests/056 | 195 +
tests/qemu-iotests/056.out | 4 +-
2 files changed, 197 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056
index
This allows us to easily force the option for all jobs belonging
to a transaction to ensure consistency with how all those jobs
will be handled.
This is purely a convenience.
Signed-off-by: John Snow
---
blockdev.c| 7 ++-
blockjob.c| 10
If speed is '0' it's not actually "less than" the previous speed.
Kick the job in this case too.
Signed-off-by: John Snow
---
blockjob.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/blockjob.c b/blockjob.c
index 3f52f29f75..24833ef30f 100644
---
This series seeks to address two distinct but closely related issues
concerning the job management API.
(1) For jobs that complete when a monitor is not attached and receiving
events or notifications, there's no way to discern the job's final
return code. Jobs must remain in the query
On 02/23/2018 11:05 AM, Kevin Wolf wrote:
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
OFFSET_VALID | DATA might be excusable because I can see that it's
convenient that a protocol driver refers to itself as *file instead of
returning NULL there and then the offset is valid (though it
Add a new state ABORTING.
This makes transitions from normative states to error states explicit
in the STM, and serves as a disambiguation for which states may complete
normally when normal end-states (CONCLUDED) are added in future commits.
Notably, Paused/Standby jobs do not transition
Expose the "manual" property via QAPI for the backup-related jobs.
As of this commit, this allows the management API to request the
"concluded" and "dismiss" semantics for backup jobs.
Signed-off-by: John Snow
---
blockdev.c | 19 ---
Some jobs upon finalization may need to perform some work that can
still fail. If these jobs are part of a transaction, it's important
that these callbacks fail the entire transaction.
We allow for a new callback in addition to commit/abort/clean that
allows us the opportunity to have fairly
This property will be used to opt-in to the new BlockJobs workflow
that allows a tighter, more explicit control over transitions from
one runstate to another.
While we're here, fix up the documentation for block_job_create
a little bit.
Signed-off-by: John Snow
---
blockjob.c
For jobs utilizing the new manual workflow, we intend to prohibit
them from modifying the block graph until the management layer provides
an explicit ACK via block-job-finalize to move the process forward.
To distinguish this runstate from "ready" or "waiting," we add a new
"pending" event.
For
Split out the pause command into the actual pause and the wait.
Not every usage presently needs to resubmit a pause request.
The intent with the next commit will be to explicitly disallow
redundant or meaningless pause/resume requests, so the tests
need to become more judicious to reflect that.
The completed_single function is getting a little mucked up with
checking to see which callbacks exist, so let's factor them out.
Signed-off-by: John Snow
---
blockjob.c | 35 ++-
1 file changed, 26 insertions(+), 9 deletions(-)
diff --git
Presently, even if a job is canceled post-completion as a result of
a failing peer in a transaction, it will still call .commit because
nothing has updated or changed its return code.
The reason why this does not cause problems currently is because
backup's implementation of .commit checks for
We're about to add several new states, and booleans are becoming
unwieldly and difficult to reason about. It would help to have a
more explicit bookkeeping of the state of blockjobs. To this end,
add a new "status" field and add our existing states in a redundant
manner alongside the bools they
Hi,
This series failed build test on s390x host. Please find the details below.
N/A. Internal error while reading log file
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-de...@freelists.org
Hi,
This series failed docker-mingw@fedora build test. Please find the testing
commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
Type: series
Message-id: 1518702707-7077-1-git-send-email-vsement...@virtuozzo.com
Subject: [Qemu-devel] [PATCH
Am 23.02.2018 um 17:43 hat Max Reitz geschrieben:
> On 2018-02-23 17:19, Kevin Wolf wrote:
> > Am 23.02.2018 um 00:25 hat Max Reitz geschrieben:
> >> On 2018-02-21 14:53, Kevin Wolf wrote:
> >>> With the conversion to a QAPI options object, the function is now
> >>> prepared to be used in a
Am 23.02.2018 um 00:10 hat Max Reitz geschrieben:
> On 2018-02-21 14:53, Kevin Wolf wrote:
> > The code to establish an RBD connection is duplicated between open and
> > create. In order to be able to share the code, factor out the code from
> > qemu_rbd_open() as a first step.
> >
> >
Am 21.02.2018 um 14:53 hat Kevin Wolf geschrieben:
> The "redundacy" option for Sheepdog image creation is currently a string
> that can encode one or two integers depending on its format, which at
> the same time implicitly selects a mode.
>
> This patch turns it into a QAPI union and converts
Am 23.02.2018 um 17:43 hat Eric Blake geschrieben:
> > OFFSET_VALID | DATA might be excusable because I can see that it's
> > convenient that a protocol driver refers to itself as *file instead of
> > returning NULL there and then the offset is valid (though it would be
> > pointless to actually
On Tue, 02/20 13:10, Stefan Hajnoczi wrote:
> 1. virtio_scsi_handle_cmd_vq() racing with iothread_stop_all() hits the
>virtio_scsi_ctx_check() assertion failure because the BDS AioContext
>has been modified by iothread_stop_all().
Does this patch fix the issue completely? IIUC
For the case where the end_transfer_func is also the caller of
ide_transfer_start, the mutual recursion can lead to unlimited
stack usage. Introduce a new version that can be used to change
tail recursion into a loop, and use it in trace_ide_atapi_cmd_reply_end.
Signed-off-by: Paolo Bonzini
The ATAPI_INT_REASON_IO interrupt is raised when I/O starts, but in the
AHCI case ide_set_irq was actually called at the end of a mutual recursion.
Move it early, with the side effect that ide_transfer_start becomes a tail
call in ide_atapi_cmd_reply_end.
Signed-off-by: Paolo Bonzini
On 02/14/2018 06:05 AM, Kevin Wolf wrote:
+static int coroutine_fn null_co_block_status(BlockDriverState *bs,
if (s->read_zeroes) {
-return BDRV_BLOCK_OFFSET_VALID | start | BDRV_BLOCK_ZERO;
-} else {
-return BDRV_BLOCK_OFFSET_VALID | start;
+ret |=
On 02/21/2018 07:53 AM, Kevin Wolf wrote:
This adds the .bdrv_co_create driver callback to gluster, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
---
qapi/block-core.json | 18 ++-
block/gluster.c | 135
On 02/21/2018 07:53 AM, Kevin Wolf wrote:
If we want to include the invalid option name in the error message, we
can't free the string earlier than that.
Signed-off-by: Kevin Wolf
---
block/rbd.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
D'oh. Should this
Am 23.02.2018 um 00:25 hat Max Reitz geschrieben:
> On 2018-02-21 14:53, Kevin Wolf wrote:
> > With the conversion to a QAPI options object, the function is now
> > prepared to be used in a .bdrv_co_create implementation.
> >
> > Signed-off-by: Kevin Wolf
> > -*s_snap =
On 2018-02-23 17:19, Kevin Wolf wrote:
> Am 23.02.2018 um 00:25 hat Max Reitz geschrieben:
>> On 2018-02-21 14:53, Kevin Wolf wrote:
>>> With the conversion to a QAPI options object, the function is now
>>> prepared to be used in a .bdrv_co_create implementation.
>>>
>>> Signed-off-by: Kevin Wolf
The callback must be invoked once we get out of the DRQ phase; because
all end_transfer_funcs end up invoking ide_transfer_stop, call it there.
While at it, remove the "notify" argument from ide_transfer_halt; the
code can simply be moved to ide_transfer_stop.
Old PATA controllers have no
There is code checking s->end_transfer_func and it was not taught
about ide_transfer_cancel. We can just use ide_transfer_stop because
s->end_transfer_func is only ever called in the DRQ phase: after
ide_transfer_cancel, the value of s->end_transfer_func is only used
as a marker and never used to
Real hardware doesn't have an unlimited stack, so the unlimited
recursion in the ATAPI code smells a bit. In fact, the call to
ide_transfer_start easily becomes a tail call with a small change
to the code (patch 4). The remaining four patches move code around
so as to the turn the call back to
Split the PIO transfer across two callbacks, thus pushing the (possibly
recursive) call to end_transfer_func up one level and out of the
AHCI-specific code.
Signed-off-by: Paolo Bonzini
---
hw/ide/ahci.c | 7 ++-
hw/ide/core.c | 9 ++---
On 02/21/2018 07:54 AM, Kevin Wolf wrote:
If bdrv_truncate() is called, but the requested size is the same as
before, don't call posix_fallocate(), which returns -EINVAL for length
zero and would therefore make bdrv_truncate() fail.
The problem can be triggered by creating a zero-sized raw
Am 23.02.2018 um 16:15 hat Eric Blake geschrieben:
> On 02/21/2018 07:53 AM, Kevin Wolf wrote:
> > If we want to include the invalid option name in the error message, we
> > can't free the string earlier than that.
> >
> > Signed-off-by: Kevin Wolf
> > ---
> > block/rbd.c | 3
All of the simple options are now passed to qcow2_create2() in a
BlockdevCreateOptions object. Still missing: node-name and the
encryption options.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
block/qcow2.c | 190
Instead of passing a separate BlockDriverState* into qcow2_create2(),
make use of the BlockdevRef that is included in BlockdevCreateOptions.
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
---
Once qcow2_create2() can be called directly on an already existing node,
we must provide the 'full' and 'falloc' preallocation modes outside of
creating the image on the protocol layer. Fortunately, we have
preallocated truncate now which can provide this functionality.
Signed-off-by: Kevin Wolf
This series implements a minimal QMP command that allows to create an
image file on the protocol level or an image format on a given block
node.
Eventually, the interface is going to change to some kind of an async
command (possibly a (non-)block job), but that will require more work on
the job
This creates a BlockdevCreateOptions union type that will contain all of
the options for image creation. We'll start out with an empty struct
type BlockdevCreateNotSupported for all drivers.
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max
Instead of passing the encryption format name and the QemuOpts down, use
the QCryptoBlockCreateOptions contained in BlockdevCreateOptions.
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
---
block/qcow2.c | 62
Currently, qcow2_create() only parses the QemuOpts and then calls
qcow2_create2() for the actual image creation, which includes both the
creation of the actual file on the file system and writing a valid empty
qcow2 image into that file.
The plan is that qcow2_create2() becomes the function that
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
---
qapi/block-core.json | 45 -
1 file changed, 44 insertions(+), 1 deletion(-)
diff --git a/qapi/block-core.json
A few block drivers will need to rename .bdrv_create options for their
QAPIfication, so let's have a helper function for that.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
include/qapi/qmp/qdict.h | 6
This allows, given a QemuOpts for a QemuOptsList that was merged from
multiple QemuOptsList, to only consider those options that exist in one
specific list. Block drivers need this to separate format-layer create
options from protocol-level options.
Signed-off-by: Kevin Wolf
This adds the .bdrv_co_create driver callback to file, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
qapi/block-core.json | 20 +-
block/file-posix.c |
Now that the options are already available in qemu_rbd_open() and not
only parsed in qemu_rbd_connect(), we can assign s->snap and
s->image_name there instead of passing the fields by reference to
qemu_rbd_connect().
Signed-off-by: Kevin Wolf
---
block/rbd.c | 14
This is almost exactly the same code. The differences are that
qemu_rbd_connect() supports BlockdevOptionsRbd.server and that the cache
mode is set explicitly.
Supporting 'server' is a welcome new feature for image creation.
Caching is disabled by default, so leave it that way.
Signed-off-by:
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
---
tests/test-qemu-opts.c | 125 +
1 file changed, 125 insertions(+)
diff --git a/tests/test-qemu-opts.c
This adds the .bdrv_co_create driver callback to file-win32, which
enables image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
block/file-win32.c | 45
Using the QAPI visitor to turn all options into QAPI BlockdevOptionsNfs
simplifies the code a lot. It will also be useful for implementing the
QAPI based .bdrv_co_create callback.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
block/nfs.c | 176
The "redundancy" option for Sheepdog image creation is currently a
string that can encode one or two integers depending on its format,
which at the same time implicitly selects a mode.
This patch turns it into a QAPI union and converts the string into such
a QAPI object before interpreting the
Create a BlockdevOptionsSsh object in connect_to_ssh() and take the
options from there. 'host_key_check' is still processed separately
because it's not in the schema yet.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
block/ssh.c | 136
Most callers have their own checks, but something like this should also
be checked centrally. As it happens, x-blockdev-create can pass negative
image sizes to format drivers (because there is no QAPI type that would
reject negative numbers) and triggers the check added by this patch.
This adds the .bdrv_co_create driver callback to ssh, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
qapi/block-core.json | 16 -
block/ssh.c | 92
Move the parsing of the QDict options up to the callers, in preparation
for the .bdrv_co_create implementation that directly gets a QAPI type.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
block/ssh.c | 34 +-
1 file
If bdrv_truncate() is called, but the requested size is the same as
before, don't call posix_fallocate(), which returns -EINVAL for length
zero and would therefore make bdrv_truncate() fail.
The problem can be triggered by creating a zero-sized raw image with
'falloc' preallocation mode.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
tests/qemu-iotests/207 | 261 +
tests/qemu-iotests/207.out | 75 +
tests/qemu-iotests/group | 1 +
3 files changed, 337 insertions(+)
Instead of manually creating the BlockdevCreateOptions object, use a
visitor to parse the given options into the QAPI object.
This involves translation from the old command line syntax to the syntax
mandated by the QAPI schema. Option names are still checked against
qcow2_create_opts, so only the
Basic test for merging two QemuOptsLists.
Signed-off-by: Kevin Wolf
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
---
tests/test-qemu-opts.c | 128 +
1 file changed, 128 insertions(+)
This adds a synchronous x-blockdev-create QMP command that can create
qcow2 images on a given node name.
We don't want to block while creating an image, so this is not the final
interface in all aspects, but BlockdevCreateOptionsQcow2 and
.bdrv_co_create() are what they actually might look like
We'll use a separate source file for image creation, and we need to
check there whether the requested driver is whitelisted.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
include/block/block.h | 1 +
This adds the .bdrv_co_create driver callback to gluster, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
qapi/block-core.json | 18 ++-
block/gluster.c | 135
This makes the host-key-check option available in blockdev-add.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
qapi/block-core.json | 63 +++--
block/ssh.c | 88
With the conversion to a QAPI options object, the function is now
prepared to be used in a .bdrv_co_create implementation.
Signed-off-by: Kevin Wolf
---
block/rbd.c | 109 +---
1 file changed, 53 insertions(+), 56
Instead of the QemuOpts in qemu_rbd_connect(), we want to use QAPI
objects. As a preparation, fetch those options directly from the QDict
that .bdrv_open() supports in the rbd driver and that are not in the
schema.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
If we want to include the invalid option name in the error message, we
can't free the string earlier than that.
Cc: qemu-sta...@nongnu.org
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
---
block/rbd.c | 3 ++-
The code to establish an RBD connection is duplicated between open and
create. In order to be able to share the code, factor out the code from
qemu_rbd_open() as a first step.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
block/rbd.c | 100
This adds the .bdrv_co_create driver callback to rbd, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
qapi/block-core.json | 19 ++-
block/rbd.c | 146
This adds the .bdrv_co_create driver callback to nfs, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
qapi/block-core.json | 16 +++-
block/nfs.c | 74
This adds the .bdrv_co_create driver callback to sheepdog, which enables
image creation over QMP.
Signed-off-by: Kevin Wolf
---
qapi/block-core.json | 24 -
block/sheepdog.c | 242 +++
2 files changed, 191 insertions(+),
Signed-off-by: Kevin Wolf
Reviewed-by: Max Reitz
---
tests/qemu-iotests/206 | 436 +
tests/qemu-iotests/206.out | 209 ++
tests/qemu-iotests/group | 1 +
3 files changed, 646
Stefan, Kevin - Ping, to take this patch. Thanks.
On 01/29/2018 07:51 AM, Stefan Hajnoczi wrote:
On Fri, Dec 15, 2017 at 04:59:13PM -0800, Deepa Srinivasan wrote:
Starting qemu with the following arguments causes qemu to segfault:
... -device lsi,id=lsi0 -drive
On 22/02/2018 18:06, John Snow wrote:
>
>
> On 02/22/2018 05:57 AM, Kevin Wolf wrote:
>> Am 20.02.2018 um 22:54 hat Paolo Bonzini geschrieben:
>>> On 20/02/2018 18:04, Peter Lieven wrote:
Hi,
I remember we discussed a long time ago to limit the stack usage of all
functions
On Thu 22 Feb 2018 08:00:08 PM CET, Eric Blake wrote:
>> One consequence of this is that even if the size field is larger than
>> it needs to be QEMU can handle it just fine: it will read more data
>> from disk but it will ignore the extra bytes.
>
> (is that true even for the corner case when the
On 2018-02-05 17:27, Max Reitz wrote:
> The issue:
>
> $ qemu-img resize -f qcow2 foo.qcow2
> qemu-img: Expecting one image file name
> Try 'qemu-img --help' for more information
>
> So we gave an image file name, but we omitted the length. qemu-img
> thinks the last argument is always
On 2018-02-14 21:49, Max Reitz wrote:
> For (x-)blockdev-create, all protocol drivers that support image
> creation also need to offer a .bdrv_truncate() implementation that
> matches in features. A previous series of mine brought gluster's and
> sheepdog's implementation up to par regarding
On 2018-02-15 14:10, Alberto Garcia wrote:
> The align_offset() function is equivalent to the ROUND_UP() macro so
> there's no need to use the former. The ROUND_UP() name is also a bit
> more explicit.
>
> This patch uses ROUND_UP() instead of the slower QEMU_ALIGN_UP()
> because align_offset()
Hi,
This series failed build test on ppcle host. Please find the details below.
Subject: [Qemu-devel] [PATCH 0/9] nbd block status base:allocation
Type: series
Message-id: 1518702707-7077-1-git-send-email-vsement...@virtuozzo.com
=== TEST SCRIPT BEGIN ===
#!/bin/bash
# Testing script will be
Hi,
iotest 203 relatively often fails for me, at least when run in parallel.
When I run the following concurrently on four shells:
$ while TEST_DIR=/tmp/t1 ./check -T -qcow2 203; do; done
$ while TEST_DIR=/tmp/t2 ./check -T -qcow2 203; do; done
$ while TEST_DIR=/tmp/t3 ./check -T -qcow2 203;
On 2018-02-12 14:14, Anton Nefedov wrote:
> This new test case only makes sense for qcow2 while iotest 033 is generic;
> however it matches the test purpose perfectly and also 033 contains those
> do_test() tricks to pass the alignment, which won't look nice being
> duplicated in other tests or
L2 entries for compressed clusters have a field that indicates the
number of sectors used to store the data in the image.
That's however not the size of the compressed data itself, just the
number of sectors where that data is located. The actual data size is
usually not a multiple of the sector
On 02/23/2018 06:50 AM, Alberto Garcia wrote:
L2 entries for compressed clusters have a field that indicates the
number of sectors used to store the data in the image.
That's however not the size of the compressed data itself, just the
number of sectors where that data is located. The actual
92 matches
Mail list logo