[PATCH 1/2] block-backend: add drained_poll

2021-05-31 Thread Sergio Lopez
Allow block backends to poll their devices/users to check if they have been quiesced when entering a drained section. This will be used in the next patch to wait for the NBD server to be completely quiesced. Suggested-by: Kevin Wolf Signed-off-by: Sergio Lopez --- block/block-backend.c

[PATCH 0/2] nbd/server: Quiesce server on drained section

2021-05-31 Thread Sergio Lopez
Before switching between AioContexts we need to make sure that we're fully quiesced ("nb_requests == 0" for every client) when entering the drained section. Otherwise, coroutines may be run in the wrong context after the switch, leading to a number of critical issues. To accomplish this, we add

[PATCH 2/2] nbd/server: Use drained block ops to quiesce the server

2021-05-31 Thread Sergio Lopez
Before switching between AioContexts we need to make sure that we're fully quiesced ("nb_requests == 0" for every client) when entering the drained section. To do this, we set "quiescing = true" for every client on ".drained_begin" to prevent new coroutines to be created, and check if

[PATCH v3] docs/secure-coding-practices: Describe how to use 'null-co' block driver

2021-05-31 Thread Philippe Mathieu-Daudé
Document that security reports must use 'null-co,read-zeroes=on' because otherwise the memory is left uninitialized (which is an on-purpose performance feature). Signed-off-by: Philippe Mathieu-Daudé --- v3: Simplified using Vladimir suggestion. --- docs/devel/secure-coding-practices.rst | 9

Re: microvm and Virtio-fs (vhost-user-fs-pci)

2021-05-31 Thread Philippe Mathieu-Daudé
Hi Paul, On 5/31/21 8:04 PM, Paul Menzel wrote: > Dear QEMU folks, > > > For minimal boot time I would like to use the machine type *microvm* > [1]. For an application, we would like to use Virtio-fs [2]. Currently > it fails with: > >     $ sudo qemu-system-x86_64 -enable-kvm -M microvm -cpu

Re: [PATCH 2/3] hw/nvme/ctrl: fix endian conversion for nsid in ctrl list

2021-05-31 Thread Klaus Jensen
On May 17 15:37, Gollu Appalanaidu wrote: In Identify Ctrl List of the CNS 0x12 and 0x13 no endian conversion for the nsid field. Signed-off-by: Gollu Appalanaidu --- hw/nvme/ctrl.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/hw/nvme/ctrl.c b/hw/nvme/ctrl.c index

Re: [PATCH 1/3] hw/nvme/ctrl: add controller list cns 0x13

2021-05-31 Thread Klaus Jensen
On May 17 15:37, Gollu Appalanaidu wrote: Add the controller identifiers list available in NVM Subsystem that may or may not be attached to namespaces. Signed-off-by: Gollu Appalanaidu --- hw/nvme/ctrl.c | 25 +++-- hw/nvme/trace-events | 2 +- include/block/nvme.h |

Re: [PATCH v2] hw/nvme/ctrl: fix csi field for cns 0x00 and 0x11

2021-05-31 Thread Klaus Jensen
On Apr 27 12:00, Gollu Appalanaidu wrote: As per the TP 4056d Namespace types CNS 0x00 and CNS 0x11 CSI field shouldn't use but it is being used for these two Identify command CNS values, fix that. Remove 'nvme_csi_has_nvm_support()' helper as suggested by Klaus we can safely assume NVM command

Re: [PATCH v3 3/5] blkdebug: track all actions

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
17.05.2021 17:50, Emanuele Giuseppe Esposito wrote: Add a counter for each action that a rule can trigger. This is mainly used to keep track of how many coroutine_yield() we need to perform after processing all rules in the list. Co-developed-by: Paolo Bonzini Signed-off-by: Emanuele Giuseppe

Re: [PATCH v3 2/5] blkdebug: move post-resume handling to resume_req_by_tag

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
17.05.2021 17:50, Emanuele Giuseppe Esposito wrote: We want to move qemu_coroutine_yield() after the loop on rules, because QLIST_FOREACH_SAFE is wrong if the rule list is modified while the coroutine has yielded. Therefore move the suspended request to the heap and clean it up from the remove

Re: [PATCH v3 1/5] blkdebug: refactor removal of a suspended request

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
17.05.2021 17:50, Emanuele Giuseppe Esposito wrote: Extract to a separate function. Do not rely on FOREACH_SAFE, which is only "safe" if the *current* node is removed---not if another node is removed. Instead, just walk the entire list from the beginning when asked to resume all suspended

Re: [PATCH] hw/nvme: add param to control auto zone transitioning to zone state closed

2021-05-31 Thread Klaus Jensen
On May 31 15:42, Niklas Cassel wrote: On Fri, May 28, 2021 at 01:22:38PM +0200, Klaus Jensen wrote: On May 28 11:05, Niklas Cassel wrote: > From: Niklas Cassel > > In the Zoned Namespace Command Set Specification, chapter > 2.5.1 Managing resources > > "The controller may transition zones in

Re: [PATCH v2 00/33] block: publish backup-top filter

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
31.05.2021 20:11, Max Reitz wrote: On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Hi all! v2: 01-02: new 03: don't bother with supporting empty child: we should never have such at this point 05: add comment 06: keep checking conflict with global add realized_set_allowed to

Re: [PATCH v2 00/33] block: publish backup-top filter

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Hi all! v2: 01-02: new 03: don't bother with supporting empty child: we should never have such at this point 05: add comment 06: keep checking conflict with global add realized_set_allowed to qdev_prop_drive_iothread 07: improve

Re: [PATCH v2 19/33] block/copy-before-write: initialize block-copy bitmap

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: We are going to publish copy-before-write filter to be used in separate of backup. Future step would support bitmap for the filter. But let's start from full set bitmap. We have to modify backup, as bitmap is first initialized by

Re: [PATCH v2 18/33] block/copy-before-write: cbw_init(): use options

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: One more step closer to .bdrv_open(): use options instead of plain arguments. Move to bdrv_open_child() calls, native for drive open handlers. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/copy-before-write.c | 40

Re: [PATCH v2 15/33] block/copy-before-write: introduce cbw_init()

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Move part of bdrv_cbw_append() to new function cbw_open(). It's an intermediate step for adding noramal .bdrv_open() handler to the Didn’t notice this in v1, but: *normal Max filter. With this commit no logic is changed, but we have a

Re: [PATCH v2 14/33] block/copy-before-write: bdrv_cbw_append(): replace child at last

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Refactor the function to replace child at last. Thus we don't need to revert it and code is simplified. block-copy state initilization being done before replacing the child still *initialization Max doesn't need any drained section.

Re: [PATCH v2 5/5] block: improve permission conflict error message

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
31.05.2021 19:35, Kevin Wolf wrote: Am 31.05.2021 um 18:18 hat Vladimir Sementsov-Ogievskiy geschrieben: 31.05.2021 19:07, Kevin Wolf wrote: Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: Now permissions are updated as follows: 1. do graph modifications ignoring

Re: [PATCH v2 5/5] block: improve permission conflict error message

2021-05-31 Thread Kevin Wolf
Am 31.05.2021 um 18:18 hat Vladimir Sementsov-Ogievskiy geschrieben: > 31.05.2021 19:07, Kevin Wolf wrote: > > Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > Now permissions are updated as follows: > > > 1. do graph modifications ignoring permissions > > > 2. do

Re: [PATCH v2 2/6] file-posix: try BLKSECTGET on block devices too, do not round to power of 2

2021-05-31 Thread Paolo Bonzini
On 31/05/21 15:59, Kevin Wolf wrote: Apparently the motivation for Maxim's patch was, if I'm reading the description correctly, that it affected non-sg cases by imposing unnecessary restrictions. I see that patch 1 changed the max_iov part so that it won't affect non-sg cases any more, but

Re: [PATCH v2 4/5] block: simplify bdrv_child_user_desc()

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
31.05.2021 18:50, Kevin Wolf wrote: Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: All existing parent types (block nodes, block devices, jobs) has the realization. So, drop unreachable code. Signed-off-by: Vladimir Sementsov-Ogievskiy Your fixes the other days showed

Re: [PATCH v2 13/33] block/copy-before-write: use file child instead of backing

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: We are going to publish copy-before-write filter, and there no public backing-child-based filter in Qemu. No reason to create a precedent, so let's refactor copy-before-write filter instead. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [PATCH v2 12/33] block/copy-before-write: drop extra bdrv_unref on failure path

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: bdrv_attach_child() do bdrv_unref() on failure, so we shouldn't do it by hand here. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/copy-before-write.c | 1 - 1 file changed, 1 deletion(-) Reviewed-by: Max Reitz

Re: [PATCH v2 2/5] block-backend: improve blk_root_get_parent_desc()

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
31.05.2021 18:45, Kevin Wolf wrote: Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: We have different types of parents: block nodes, block backends and jobs. So, it makes sense to specify type together with name. While being here also use g_autofree. iotest 307 output is

Re: [PATCH v2 5/5] block: improve permission conflict error message

2021-05-31 Thread Vladimir Sementsov-Ogievskiy
31.05.2021 19:07, Kevin Wolf wrote: Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: Now permissions are updated as follows: 1. do graph modifications ignoring permissions 2. do permission update (of course, we rollback [1] if [2] fails) So, on stage [2] we can't say

Re: [PATCH v2 07/33] block: rename backup-top to copy-before-write

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: We are going to convert backup_top to full featured public filter, which can be used in separate of backup job. Start from renaming from "how it used" to "what it does". While updating comments in 283 iotest, drop and rephrase also things

Re: [PATCH v2 5/5] block: improve permission conflict error message

2021-05-31 Thread Kevin Wolf
Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: > Now permissions are updated as follows: > 1. do graph modifications ignoring permissions > 2. do permission update > > (of course, we rollback [1] if [2] fails) > > So, on stage [2] we can't say which users are "old" and

Re: [PATCH v2 06/33] qdev: allow setting drive property for realized device

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: We need an ability to insert filters above top block node, attached to block device. It can't be achieved with blockdev-reopen command. So, we want do it with help of qom-set. Intended usage: Assume there is a node A that is attached to

Re: [PATCH v2 05/33] qdev-properties: PropertyInfo: add realized_set_allowed field

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Add field, so property can declare support for setting the property when device is realized. To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/hw/qdev-properties.h | 1 +

Re: [PATCH v2 4/5] block: simplify bdrv_child_user_desc()

2021-05-31 Thread Kevin Wolf
Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: > All existing parent types (block nodes, block devices, jobs) has the > realization. So, drop unreachable code. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Your fixes the other days showed that vvfat has a BdrvChildClass,

Re: [PATCH v2 2/5] block-backend: improve blk_root_get_parent_desc()

2021-05-31 Thread Kevin Wolf
Am 04.05.2021 um 11:45 hat Vladimir Sementsov-Ogievskiy geschrieben: > We have different types of parents: block nodes, block backends and > jobs. So, it makes sense to specify type together with name. > > While being here also use g_autofree. > > iotest 307 output is updated. > >

Re: [PATCH] hw/nvme: add param to control auto zone transitioning to zone state closed

2021-05-31 Thread Niklas Cassel
On Fri, May 28, 2021 at 01:22:38PM +0200, Klaus Jensen wrote: > On May 28 11:05, Niklas Cassel wrote: > > From: Niklas Cassel > > > > In the Zoned Namespace Command Set Specification, chapter > > 2.5.1 Managing resources > > > > "The controller may transition zones in the ZSIO:Implicitly Opened

Re: [PATCH v2 0/3] block: drop BlockDriverState::read_only

2021-05-31 Thread Kevin Wolf
Am 27.05.2021 um 17:40 hat Vladimir Sementsov-Ogievskiy geschrieben: > Hi all! > > The field duplicates information in .open_flags. We have to carefully > sync these two fields everywhere. It's simple to introduce a bug by > forgetting it. > > Let's drop the field, and fix users to call

Re: [PATCH v2 4/5] progressmeter: protect with a mutex

2021-05-31 Thread Stefan Hajnoczi
On Tue, May 18, 2021 at 12:14:17PM +0200, Emanuele Giuseppe Esposito wrote: > On 18/05/2021 12:00, Vladimir Sementsov-Ogievskiy wrote: > > 18.05.2021 12:40, Emanuele Giuseppe Esposito wrote: > > > Progressmeter is protected by the AioContext mutex, which > > > is taken by the block jobs and their

Re: [PATCH v2 03/33] block: introduce bdrv_replace_child_bs()

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Add function to transactionally replace bs inside BdrvChild. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 2 ++ block.c | 31 +++ 2 files changed, 33 insertions(+)

Re: [PATCH v2 2/6] file-posix: try BLKSECTGET on block devices too, do not round to power of 2

2021-05-31 Thread Kevin Wolf
Am 27.05.2021 um 22:14 hat Paolo Bonzini geschrieben: > On 27/05/21 17:51, Kevin Wolf wrote: > > Am 24.05.2021 um 18:36 hat Paolo Bonzini geschrieben: > > > bs->sg is only true for character devices, but block devices can also > > > be used with scsi-block and scsi-generic. Unfortunately

Re: [RFC PATCH] block/vpc: Support probing of fixed-size VHD images

2021-05-31 Thread Max Reitz
On 19.05.21 12:19, Thomas Huth wrote: On 29/03/2021 09.25, Thomas Huth wrote: Fixed-size VHD images don't have a header, only a footer. To be able to still detect them right, support probing via the file name, too. Without this change, images get detected as raw: $ qemu-img create -f vpc -o

Re: [PATCH v2 02/33] block: comment graph-modifying function not updating permissions

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 7 +++ 1 file changed, 7 insertions(+) Reviewed-by: Max Reitz

Re: [PATCH v2 01/33] block: rename bdrv_replace_child to bdrv_replace_child_tran

2021-05-31 Thread Max Reitz
On 20.05.21 16:21, Vladimir Sementsov-Ogievskiy wrote: We have bdrv_replace_child() wrapper on bdrv_replace_child_noperm(). But bdrv_replace_child() doesn't update permissions. It's rather strange, as normally it's expected that foo() should call foo_noperm() and update permissions. Let's

Re: [PULL 00/44] Python patches

2021-05-31 Thread Philippe Mathieu-Daudé
On 5/30/21 9:22 PM, John Snow wrote: > On 5/30/21 3:09 PM, Peter Maydell wrote: >> Fails to build on my machine that runs the BSD VMs, apparently >> before it gets to the point of launching the VM: >> >> make: Entering directory '/home/peter.maydell/qemu-freebsd/build' >> /usr/bin/python3 -B