[...]
>
> -def qmp(self, cmd, conv_keys=True, **args):
> -"""
> -Invoke a QMP command and return the response dict
> -"""
> +@classmethod
> +def _qmp_args(cls, _conv_keys: bool = True, **args: Any) -> Dict[str,
> Any]:
> qmp_args = dict()
>
Logical and physical block sizes in QEMU are limited to 32 KiB.
This appears unnecessarily tight, and we've seen bigger block sizes
handy at times.
Lift the limitation up to 2 MiB which appears to be good enough for
everybody, and matches the qcow2 cluster size limit.
Signed-off-by: Roman Kagan
Convert all size-related properties in BlockConf to 32bit. This will
accommodate bigger block sizes (in a followup patch). This also allows
to make them all accept size suffixes, either via DEFINE_PROP_BLOCKSIZE
or via DEFINE_PROP_SIZE32.
Also, since min_io_size is exposed to the guest by scsi a
Several block device properties related to blocksize configuration must
be in certain relationship WRT each other: physical block must be no
smaller than logical block; min_io_size, opt_io_size, and
discard_granularity must be a multiple of a logical block.
To ensure these requirements are met, ad
Add getter for size32, and use it for blocksize, too.
In its human-readable branch, it reports approximate size in
human-readable units next to the exact byte value, like the getter for
64bit size does.
Adjust the expected test output accordingly.
Signed-off-by: Roman Kagan
Reviewed-by: Eric Bl
Introduce size32 property type which handles size suffixes (k, m, g)
just like size property, but is uint32_t rather than uint64_t. It's
going to be useful for properties that are byte sizes but are inherently
32bit, like BlkConf.opt_io_size or .discard_granularity (they are
switched to this new p
It appears convenient to be able to specify physical_block_size and
logical_block_size using common size suffixes.
Teach the blocksize property setter to interpret them. Also express the
upper and lower limits in the respective units.
Signed-off-by: Roman Kagan
Reviewed-by: Eric Blake
---
hw/
BlockConf includes several properties counted in bytes.
Enhance their handling in some aspects, specifically
- accept common size suffixes (k, m)
- perform consistency checks on the values
- lift the upper limit on physical_block_size and logical_block_size
Also fix the accessor for opt_io_size
The width of opt_io_size in virtio_blk_config is 32bit. However, it's
written with virtio_stw_p; this may result in value truncation, and on
big-endian systems with legacy virtio in completely bogus readings in
the guest.
Use the appropriate accessor to store it.
Signed-off-by: Roman Kagan
Revi
Make it easier (more visible) to maintain the limits on the blocksize
properties in sync with the respective description, by using macros both
in the code and in the description.
Signed-off-by: Roman Kagan
Reviewed-by: Eric Blake
---
hw/core/qdev-properties.c | 21 +++--
1 file
On Thu, May 28, 2020 at 04:45:19PM -0500, Eric Blake wrote:
> On 5/28/20 4:39 PM, Roman Kagan wrote:
> > Introduce size32 property type which handles size suffixes (k, m) just
> > like size property, but is uint32_t rather than uint64_t.
>
> Does it handle 'g' as well? (even though the set of vali
On 5/28/20 4:39 PM, Roman Kagan wrote:
Add getter for size32, and use it for blocksize, too.
In its human-readable branch, it reports approximate size in
human-readable units next to the exact byte value, like the getter for
64bit size does.
Adjust the expected test output accordingly.
Signed-
On 5/28/20 4:39 PM, Roman Kagan wrote:
Convert all size-related properties in BlockConf to 32bit. This will
allow to accomodate bigger block sizes (in a followup patch).
s/allow to accomodate/accommodate/
This also allows to make them all accept size suffixes, either via
DEFINE_PROP_BLOCKSIZ
On 5/28/20 4:39 PM, Roman Kagan wrote:
Introduce size32 property type which handles size suffixes (k, m) just
like size property, but is uint32_t rather than uint64_t.
Does it handle 'g' as well? (even though the set of valid 32-bit sizes
with a g suffix is rather small ;)
It's going to
be
On 5/28/20 4:39 PM, Roman Kagan wrote:
It appears convenient to be able to specify physical_block_size and
logical_block_size using common size suffixes.
Teach the blocksize property setter to interpret them. Also express the
upper and lower limits in the respective units.
Signed-off-by: Roman
Convert all size-related properties in BlockConf to 32bit. This will
allow to accomodate bigger block sizes (in a followup patch).
This also allows to make them all accept size suffixes, either via
DEFINE_PROP_BLOCKSIZE or via DEFINE_PROP_SIZE32.
Also, since min_io_size is exposed to the guest by
Add getter for size32, and use it for blocksize, too.
In its human-readable branch, it reports approximate size in
human-readable units next to the exact byte value, like the getter for
64bit size does.
Adjust the expected test output accordingly.
Signed-off-by: Roman Kagan
---
v6 -> v7:
- spli
Logical and physical block sizes in QEMU are limited to 32 KiB.
This appears unnecessarily tight, and we've seen bigger block sizes
handy at times.
Lift the limitation up to 2 MiB which appears to be good enough for
everybody, and matches the qcow2 cluster size limit.
Signed-off-by: Roman Kagan
Make it easier (more visible) to maintain the limits on the blocksize
properties in sync with the respective description, by using macros both
in the code and in the description.
Signed-off-by: Roman Kagan
Reviewed-by: Eric Blake
---
v4 -> v5:
- split out into separate patch [Philippe]
hw/core
BlockConf includes several properties counted in bytes.
Enhance their handling in some aspects, specifically
- accept common size suffixes (k, m)
- perform consistency checks on the values
- lift the upper limit on physical_block_size and logical_block_size
Also fix the accessor for opt_io_size
Several block device properties related to blocksize configuration must
be in certain relationship WRT each other: physical block must be no
smaller than logical block; min_io_size, opt_io_size, and
discard_granularity must be a multiple of a logical block.
To ensure these requirements are met, ad
It appears convenient to be able to specify physical_block_size and
logical_block_size using common size suffixes.
Teach the blocksize property setter to interpret them. Also express the
upper and lower limits in the respective units.
Signed-off-by: Roman Kagan
---
v6 -> v7:
- split out into se
Introduce size32 property type which handles size suffixes (k, m) just
like size property, but is uint32_t rather than uint64_t. It's going to
be useful for properties that are byte sizes but are inherently 32bit,
like BlkConf.opt_io_size or .discard_granularity (they are switched to
this new prop
The width of opt_io_size in virtio_blk_config is 32bit. However, it's
written with virtio_stw_p; this may result in value truncation, and on
big-endian systems with legacy virtio in completely bogus readings in
the guest.
Use the appropriate accessor to store it.
Signed-off-by: Roman Kagan
Revi
On 5/28/20 10:04 AM, Alberto Garcia wrote:
On Wed 27 May 2020 07:58:10 PM CEST, Eric Blake wrote:
There is just one thing to take into account for a possible future
improvement: compressed clusters cannot be partially zeroized so
zero_l2_subclusters() on the head or the tail can return -ENOTSUP.
On 5/28/20 10:37 AM, Kevin Wolf wrote:
This way, a monitor command handler will still be able to access the
current monitor, but when it yields, all other code code will correctly
get NULL from monitor_cur().
Outside of coroutine context, qemu_coroutine_self() returns the leader
coroutine of the
On 5/28/20 10:37 AM, Kevin Wolf wrote:
The correct way to set the current monitor for a coroutine handler is
different that for a blocking handler, so monitor_set_cur() can only be
s/that/than/
called in qmp_dispatch().
Signed-off-by: Kevin Wolf
---
include/qapi/qmp/dispatch.h | 3 ++-
m
On 5/28/20 10:37 AM, Kevin Wolf wrote:
monitor_qmp_dispatch() is never supposed to be called in the context of
another monitor, so assert that monitor_cur() is NULL instead of saving
and restoring it.
Signed-off-by: Kevin Wolf
---
monitor/qmp.c | 5 ++---
1 file changed, 2 insertions(+), 3 d
On 5/28/20 10:37 AM, Kevin Wolf wrote:
cur_mon is updated relatively early in the command handling code even
though only the command handler actually needs it. Move it to
handle_hmp_command().
Signed-off-by: Kevin Wolf
---
monitor/hmp.c | 23 ---
monitor/misc.c | 7
On 5/28/20 10:37 AM, Kevin Wolf wrote:
cur_mon really needs to be coroutine-local as soon as we move monitor
command handlers to coroutines and let them yield. As a first step, just
remove all direct accesses to cur_mon so that we can implement this in
the getter function later.
Signed-off-by: K
On 5/28/20 10:37 AM, Kevin Wolf wrote:
cur_mon really needs to be coroutine-local as soon as we move monitor
command handlers to coroutines and let them yield. As a first step, just
remove all direct accesses to cur_mon so that we can implement this in
the getter function later.
Signed-off-by: K
On 5/28/20 10:37 AM, Kevin Wolf wrote:
Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_set_cpu().
Signed-off-by: Kevin Wolf
---
include/monitor/monitor.h | 2 +-
monitor/hmp-cmds.c| 2 +-
monitor/misc.c| 10
It's useful to know how much space can be occupied by qcow2 persistent
bitmaps, even though such metadata is unrelated to the guest-visible
data. Report this value as an additional QMP field, present when
measuring an existing image and output format that both support
bitmaps. Update iotest 178 a
28.05.2020 20:09, Stefan Hajnoczi wrote:
On Thu, May 28, 2020 at 01:15:02PM +0300, Vladimir Sementsov-Ogievskiy wrote:
Thanks to Eric, the whole series is reviewed now!
v4:
01: fix grammar in comment, add Eric's r-b
02-05: add Eric's r-b
Hi all!
These series are here to address the following
28.05.2020 18:17, Stefan Hajnoczi wrote:
On Wed, May 20, 2020 at 05:49:01PM +0300, Vladimir Sementsov-Ogievskiy wrote:
We have a few bdrv_*() functions that can either spawn a new coroutine
and wait for it with BDRV_POLL_WHILE() or use a fastpath if they are
alreeady running in a coroutine. All
On Thu, May 28, 2020 at 01:15:02PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> Thanks to Eric, the whole series is reviewed now!
>
> v4:
> 01: fix grammar in comment, add Eric's r-b
> 02-05: add Eric's r-b
>
> Hi all!
>
> These series are here to address the following problem:
> block-status-ab
On 5/14/20 6:45 AM, Klaus Jensen wrote:
> From: Klaus Jensen
>
> If the device uses MSI-X, any of the 2048 MSI-X interrupt vectors are
> valid. If the device is not using MSI-X, vector will and can only be
> zero at this point.
>
> Cc: "Michael S. Tsirkin"
> Cc: Marcel Apfelbaum
> Signed-off-b
Add a function that can be used to move the currently running coroutine
to a different AioContext (and therefore potentially a different
thread).
Signed-off-by: Kevin Wolf
---
include/block/aio.h | 10 ++
util/async.c| 30 ++
2 files changed, 40 insert
Often, QMP command handlers are not only called to handle QMP commands,
but also from a corresponding HMP command handler. In order to give them
a consistent environment, optionally run HMP command handlers in a
coroutine, too.
The implementation is a lot simpler than in QMP because for HMP, we
st
block_resize performs some I/O that could potentially take quite some
time, so use it as an example for the new 'coroutine': true annotation
in the QAPI schema.
bdrv_truncate() requires that we're already in the right AioContext for
the BlockDriverState if called in coroutine context. So instead o
This patch adds a new 'coroutine' flag to QMP command definitions that
tells the QMP dispatcher that the command handler is safe to be run in a
coroutine.
The documentation of the new flag pretends that this flag is already
used as intended, which it isn't yet after this patch. We'll implement
thi
This moves the QMP dispatcher to a coroutine and runs all QMP command
handlers that declare 'coroutine': true in coroutine context so they
can avoid blocking the main loop while doing I/O or waiting for other
events.
For commands that are not declared safe to run in a coroutine, the
dispatcher dro
Add a function to move the current coroutine to the AioContext of a
given BlockDriverState.
Signed-off-by: Kevin Wolf
---
include/block/block.h | 6 ++
block.c | 10 ++
2 files changed, 16 insertions(+)
diff --git a/include/block/block.h b/include/block/block.h
index
The correct way to set the current monitor for a coroutine handler is
different that for a blocking handler, so monitor_set_cur() can only be
called in qmp_dispatch().
Signed-off-by: Kevin Wolf
---
include/qapi/qmp/dispatch.h | 3 ++-
monitor/qmp.c | 7 +--
qapi/qmp-dispatch.c
Most callers actually don't have to rely on cur_mon, but already know
for which monitor they call monitor_set_cpu().
Signed-off-by: Kevin Wolf
---
include/monitor/monitor.h | 2 +-
monitor/hmp-cmds.c| 2 +-
monitor/misc.c| 10 +-
3 files changed, 7 insertions(+), 7
monitor_qmp_dispatch() is never supposed to be called in the context of
another monitor, so assert that monitor_cur() is NULL instead of saving
and restoring it.
Signed-off-by: Kevin Wolf
---
monitor/qmp.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/monitor/qmp.c b/m
cur_mon is updated relatively early in the command handling code even
though only the command handler actually needs it. Move it to
handle_hmp_command().
Signed-off-by: Kevin Wolf
---
monitor/hmp.c | 23 ---
monitor/misc.c | 7 ---
2 files changed, 12 insertions(+), 18
This way, a monitor command handler will still be able to access the
current monitor, but when it yields, all other code code will correctly
get NULL from monitor_cur().
Outside of coroutine context, qemu_coroutine_self() returns the leader
coroutine of the current thread.
Signed-off-by: Kevin Wo
Some QMP command handlers can block the main loop for a relatively long
time, for example because they perform some I/O. This is quite nasty.
Allowing such handlers to run in a coroutine where they can yield (and
therefore release the BQL) while waiting for an event such as I/O
completion solves th
cur_mon really needs to be coroutine-local as soon as we move monitor
command handlers to coroutines and let them yield. As a first step, just
remove all direct accesses to cur_mon so that we can implement this in
the getter function later.
Signed-off-by: Kevin Wolf
---
include/monitor/monitor.h
On Wed, 27 May 2020 11:29:21 +0100
Stefan Hajnoczi wrote:
> Multi-queue devices achieve the best performance when each vCPU has a
> dedicated queue. This ensures that virtqueue used notifications are
> handled on the same vCPU that submitted virtqueue buffers. When another
> vCPU handles the the
On Tue, May 26, 2020 at 05:20:24PM +0200, Philippe Mathieu-Daudé wrote:
> On 5/26/20 4:55 PM, Philippe Mathieu-Daudé wrote:
> > On 5/19/20 7:11 PM, Stefan Hajnoczi wrote:
> >> Passing around both BDRVNVMeState and NVMeQueuePair is unwiedly. Reduce
>
> Oh, and typo "unwieldy".
Thanks, will fix!
S
On Mon, May 25, 2020 at 10:07:13AM +0200, Sergio Lopez wrote:
> On Tue, May 19, 2020 at 06:11:32PM +0100, Stefan Hajnoczi wrote:
> > A lot of CPU time is spent simply locking/unlocking q->lock during
> > polling. Check for completion outside the lock to make q->lock disappear
> > from the profile.
On 5/27/20 12:29 PM, Stefan Hajnoczi wrote:
> The event and control virtqueues are always present, regardless of the
> multi-queue configuration. Define a constant so that virtqueue number
> calculations are easier to read.
>
> Signed-off-by: Stefan Hajnoczi
> Reviewed-by: Cornelia Huck
> ---
>
On Wed, May 20, 2020 at 05:49:01PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> We have a few bdrv_*() functions that can either spawn a new coroutine
> and wait for it with BDRV_POLL_WHILE() or use a fastpath if they are
> alreeady running in a coroutine. All of them duplicate basically the
> same
On Wed 27 May 2020 07:58:10 PM CEST, Eric Blake wrote:
>> There is just one thing to take into account for a possible future
>> improvement: compressed clusters cannot be partially zeroized so
>> zero_l2_subclusters() on the head or the tail can return -ENOTSUP.
>> This makes the caller repeat the
> Automatically size the number of virtio-blk-pci request virtqueues to
> match the number of vCPUs. Other transports continue to default to 1
> request virtqueue.
>
> A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are
> handled on the same vCPU that submitted the request. No IPI
> The event and control virtqueues are always present, regardless of the
> multi-queue configuration. Define a constant so that virtqueue number
> calculations are easier to read.
>
> Signed-off-by: Stefan Hajnoczi
> Reviewed-by: Cornelia Huck
> ---
> include/hw/virtio/virtio-scsi.h | 3 +++
>
In order to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above, let's support include_base parameter.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Kevin Wolf
Reviewed-by: Eric Blake
---
block/io.c | 19 ++-
1 file changed, 14 insertions(+), 5 deletions(
Thanks to Eric, the whole series is reviewed now!
v4:
01: fix grammar in comment, add Eric's r-b
02-05: add Eric's r-b
Hi all!
These series are here to address the following problem:
block-status-above functions may consider space after EOF of
intermediate backing files as unallocated, which is
These cases are fixed by previous patches around block_status and
is_allocated.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
---
tests/qemu-iotests/274 | 20
tests/qemu-iotests/274.out | 65 ++
2 files changed, 85 inser
bdrv_co_block_status_above has several design problems with handling
short backing files:
1. With want_zeros=true, it may return ret with BDRV_BLOCK_ZERO but
without BDRV_BLOCK_ALLOCATED flag, when actually short backing file
which produces these after-EOF zeros is inside requested backing
sequenc
bdrv_is_allocated_above wrongly handles short backing files: it reports
after-EOF space as UNALLOCATED which is wrong, as on read the data is
generated on the level of short backing file (if all overlays has
unallocated area at that place).
Reusing bdrv_common_block_status_above fixes the issue an
We are going to reuse bdrv_common_block_status_above in
bdrv_is_allocated_above. bdrv_is_allocated_above may be called with
include_base == false and still bs == base (for ex. from img_rebase()).
So, support this corner case.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Kevin Wolf
R
Currently this field only set by qed and qcow2. But in fact, all
backing-supporting formats (parallels, qcow, qcow2, qed, vmdk) share
these semantics: on unallocated blocks, if there is no backing file they
just memset the buffer with zeroes.
So, document this behavior for .supports_backing and dr
We set bdi->unallocated_blocks_are_zero = iscsilun->lbprz, but
iscsi_co_block_status doesn't return 0 in case of iscsilun->lbprz, it
returns ZERO when appropriate. So actually unallocated_blocks_are_zero
is useless (it doesn't affect the only user of the field:
bdrv_co_block_status()). Drop it now.
From: Eric Blake
The other four drivers that support backing files (qcow, qcow2,
parallels, vmdk) all rely on the block layer to populate zeroes when
reading beyond EOF of a short backing file. We can simplify the qed
code by doing likewise.
Signed-off-by: Eric Blake
Reviewed-by: Vladimir Seme
vhdx doesn't have .bdrv_co_block_status handler, so DATA|ALLOCATED is
always assumed for it in bdrv_co_block_status().
unallocated_blocks_are_zero is useless (it doesn't affect the only user
of the field: bdrv_co_block_status()), drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: E
In case when get_image_offset() returns -1, we do zero out the
corresponding chunk of qiov. So, this should be reported as ZERO.
Note that this changes visible output of "qemu-img map --output=json"
and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what i
In case of !VDI_IS_ALLOCATED[], we do zero out the corresponding chunk
of qiov. So, this should be reported as ZERO.
Note that this changes visible output of "qemu-img map --output=json"
and "qemu-io -c map" commands. For qemu-img map, the change is obvious:
we just mark as zero what is really zer
raw_co_block_status() in block/file-posix.c never returns 0, so
unallocated_blocks_are_zero is useless (it doesn't affect the only user
of the field: bdrv_co_block_status()). Drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
---
block/file-posix.c | 3 ---
1 file chan
This is first step to block-status refactoring, and solves most simple
problem mentioned in my investigation of block-status described in
the thread "backing chain & block status & filters":
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg04706.html
The whole series is reviewed, let's p
It's false by default, no needs to set it. We are going to drop this
variable at all, so drop it now here, it doesn't hurt.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
---
block/crypto.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/block/crypto.c b/block/crypto.c
i
qemu-img convert wants to distinguish ZERO which comes from short
backing files. unallocated_blocks_are_zero field of bdi is unrelated:
space after EOF is always considered to be zero anyway. So, just make
post_backing_zero true in case of short backing file.
Signed-off-by: Vladimir Sementsov-Ogie
The function has only one user: bdrv_co_block_status(). Inline it to
simplify reviewing of the following patches, which will finally drop
unallocated_blocks_are_zero field too.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Eric Blake
---
include/block/block.h | 1 -
block.c
07.05.2020 17:08, Eric Blake wrote:
On 5/7/20 3:47 AM, Vladimir Sementsov-Ogievskiy wrote:
The function has the only user: bdrv_co_block_status(). Inline it to
s/the only/only one/
simplify reviewing of the following patches, which will finally drop
unallocated_blocks_are_zero field too.
Si
A socket write during vhost-user communication may trigger a disconnect
event, calling vhost_user_blk_disconnect() and clearing all the
vhost_dev structures holding data that vhost-user functions expect to
remain valid to roll back initialization correctly. Delay the cleanup to
keep vhost_dev struc
Changes is v4:
- Update the "[PATCH v4 2/2] vhost-user-blk: delay
vhost_user_blk_disconnect" patch based on Raphael's comment and Li
Feng previous commit:
https://lists.gnu.org/archive/html/qemu-devel/2020-04/msg02255.html
Don't change the vhost_user_blk_device_realize() function. Update th
During testing of the vhost-user-blk reconnect functionality the qemu
SIGSEGV was triggered:
start qemu as:
x86_64-softmmu/qemu-system-x86_64 -m 1024M -M q35 \
-object
memory-backend-file,id=ram-node0,size=1024M,mem-path=/dev/shm/qemu,share=on \
-numa node,cpus=0,memdev=ram-node0 \
-cha
On Wed, May 27, 2020 at 11:38:17AM +0100, Daniel P. Berrangé wrote:
> On Wed, May 27, 2020 at 11:29:23AM +0100, Stefan Hajnoczi wrote:
> > Automatically size the number of virtio-scsi-pci, vhost-scsi-pci, and
> > vhost-user-scsi-pci request virtqueues to match the number of vCPUs.
> > Other transpo
> -Original Message-
> From: Roman Kagan
> Sent: 27 May 2020 13:45
> To: qemu-de...@nongnu.org
> Cc: Kevin Wolf ; xen-de...@lists.xenproject.org; Gerd
> Hoffmann ;
> Daniel P. Berrangé ; Paolo Bonzini
> ; Anthony Perard
> ; Laurent Vivier ; Max Reitz
> ; qemu-
> bl...@nongnu.org; Philip
81 matches
Mail list logo
