[PATCH] hw/block/nvme: add compare command

2020-11-23 Thread Klaus Jensen
From: Gollu Appalanaidu Add the Compare command. This implementation uses a bounce buffer to read in the data from storage and then compare with the host supplied buffer. Signed-off-by: Gollu Appalanaidu [k.jensen: rebased] Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 100

[PATCH 2/2] hw/block/nvme: add simple copy command

2020-11-23 Thread Klaus Jensen
From: Klaus Jensen Add support for TP 4065a ("Simple Copy Command"), v2020.05.04 ("Ratified"). The implementation uses a bounce buffer to first read in the source logical blocks, then issue a write of that bounce buffer. The default maximum number of source logical blocks is 128, translating to

[PATCH 1/2] nvme: updated shared header for copy command

2020-11-23 Thread Klaus Jensen
From: Klaus Jensen Add new data structures and types for the Simple Copy command. Signed-off-by: Klaus Jensen Cc: Stefan Hajnoczi Cc: Fam Zheng --- include/block/nvme.h | 45 ++-- 1 file changed, 43 insertions(+), 2 deletions(-) diff --git

[PATCH 0/2] hw/block/nvme: add simple copy command

2020-11-23 Thread Klaus Jensen
From: Klaus Jensen Add support for TP 4065 ("Simple Copy Command"). Klaus Jensen (2): nvme: updated shared header for copy command hw/block/nvme: add simple copy command hw/block/nvme-ns.h| 4 + hw/block/nvme.h | 1 + include/block/nvme.h | 45 - hw/block/nvme-ns.c

Re: [PATCH v8 0/5] hw/block/nvme: dulbe and dsm support

2020-11-23 Thread Klaus Jensen
On Nov 12 20:59, Klaus Jensen wrote: > From: Klaus Jensen > > This adds support for the Deallocated or Unwritten Logical Block error > recovery feature as well as the Dataset Management command. > > v8: > - Move req->opaque clearing to nvme_req_clear. > - Add two preparation/cleanup

[PATCH 19/21] block: add bdrv_remove_backing transaction action

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 42 -- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/block.c b/block.c index cf7b859a81..6ea926e2c1 100644 --- a/block.c +++ b/block.c @@ -2974,12 +2974,19 @@ BdrvChild

[PATCH 21/21] block/backup-top: drop .active

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
We don't need this workaround anymore: bdrv_append is already smart enough and we can use new bdrv_drop_filter(). Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/backup-top.c | 38 +- tests/qemu-iotests/283.out | 2 +- 2 files changed, 2

[PATCH 20/21] block: introduce bdrv_drop_filter()

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Using bdrv_replace_node() for removing filter is not good enough: it keeps child reference of the filter, which may conflict with original top node during permission update. Instead let's create new interface, which will do all graph modifications first and then update permissions. Let's modify

[PATCH 09/21] block: add bdrv_drv_set_perm transaction action

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Refactor calling driver callbacks to a separate transaction action to be used later. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 71 - 1 file changed, 55 insertions(+), 16 deletions(-) diff --git a/block.c b/block.c index

[PATCH 17/21] block: bdrv_append(): return status

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Return int status to avoid extra error propagation schemes. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 4 ++-- block.c | 15 --- block/commit.c | 6 ++ block/mirror.c | 6 ++ blockdev.c

[PATCH 06/21] block: rewrite bdrv_child_try_set_perm() using bdrv_refresh_perms()

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
We are going to drop recursive bdrv_child_* functions, so stop use them in bdrv_child_try_set_perm() as a first step. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/block.c b/block.c index

[PATCH 02/21] tests/test-bdrv-graph-mod: add test_parallel_perm_update

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Add test to show that simple DFS recursion order is not correct for permission update. Correct order is topological-sort order, which will be introduced later. Consider the block driver which has two filter children: one active with exclusive write access and one inactive with no specific

[PATCH 16/21] block: bdrv_append(): don't consume reference

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
We have too much comments for this feature. It seems better just don't do it. Most of real users (tests don't count) have to create additional reference. Drop also comment in external_snapshot_prepare: - bdrv_append doesn't "remove" old bs in common sense, it sounds strange - the fact that

[PATCH 13/21] block: fix bdrv_replace_node_common

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
inore_children thing doesn't help to track all propagated permissions of children we want to ignore. The simplest way to correctly update permissions is update graph first and then do permission update. In this case we just referesh permissions for the whole subgraph (in topological-sort defined

[PATCH 12/21] block: return value from bdrv_replace_node()

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Functions with errp argument are not recommened to be void-functions. Improve bdrv_replace_node(). Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/block/block.h | 4 ++-- block.c | 14 -- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git

[PATCH 11/21] block: add bdrv_replace_child_safe() transaction action

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
To be used in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 54 ++ 1 file changed, 54 insertions(+) diff --git a/block.c b/block.c index d799afeedd..1b10b6fb5e 100644 --- a/block.c +++ b/block.c @@ -82,6

[PATCH 15/21] block: split out bdrv_replace_node_noperm()

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Split part of bdrv_replace_node_common() to be used separately. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 47 ++- 1 file changed, 30 insertions(+), 17 deletions(-) diff --git a/block.c b/block.c index 5f6ad1d016..1327254b8e 100644 ---

[PATCH 14/21] block: add bdrv_attach_child_noperm() transaction action

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
The code partly duplicates bdrv_root_attach_child() and bdrv_attach_child(). Still refactoring these two functions by renaming them to *_common with new noperm argument is more complicating. When all operations moved to new graph update paradigm (update permissions only on updated graph) all

[PATCH 1/2] block: make bdrv_drop_intermediate() less wrong

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
First, permission update loop tries to do iterations transactionally, but the whole update is not transactional: nobody roll-back successful loop iterations when some iteration fails. Second, in the iteration we have nested permission update: c->klass->update_filename may point to

[PATCH 18/21] block: adapt bdrv_append() for inserting filters

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
bdrv_append is not very good for inserting filters: it does extra permission update as part of bdrv_set_backing_hd(). During this update filter may conflict with other parents of top_bs. Instead, let's first do all graph modifications and after it update permissions. Note: bdrv_append() is still

[PATCH 08/21] block: use topological sort for permission update

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Rewrite bdrv_check_perm(), bdrv_abort_perm_update() and bdrv_set_perm() to update nodes in topological sort order instead of simple DFS. With topologically sorted nodes, we update a node only when all its parents already updated. With DFS it's not so. Consider the following example: A -+

[PATCH 07/21] block: inline bdrv_child_*() permission functions calls

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Each of them has only one caller. Open-coding simplifies further pemission-update system changes. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 59 + 1 file changed, 17 insertions(+), 42 deletions(-) diff --git a/block.c

[PATCH 10/21] block: add bdrv_list_* permission update functions

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Add new interface, allowing use of existing node list. It will be used to fix bdrv_replace_node() in the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 112 ++-- 1 file changed, 76 insertions(+), 36 deletions(-)

[PATCH 05/21] block: refactor bdrv_child* permission functions

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Split out non-recursive parts, and refactor as block graph transaction action. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 79 ++--- 1 file changed, 59 insertions(+), 20 deletions(-) diff --git a/block.c b/block.c index

[PATCH 04/21] block: bdrv_refresh_perms: check parents compliance

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Add additional check that node parents do not interfere with each other. This should not hurt existing callers and allows in further patch use bdrv_refresh_perms() to update a subtree of changed BdrvChild (check that change is correct). New check will substitute bdrv_check_update_perm() in

[PATCH 03/21] util: add transactions.c

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Add simple transaction API to use in further update of block graph operations. Supposed usage is: - "prepare" is main function of the action and it should make the main effect of the action to be visible for the following actions, keeping possibility of roll-back, saving necessary things in

[PATCH 01/21] tests/test-bdrv-graph-mod: add test_parallel_exclusive_write

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Add the test that shows that concept of ignore_children is incomplete. Actually, when we want to update something, ignoring permission of some existing BdrvChild, we should ignore also the propagated effect of this child to the other children. But that's not done. Better approach (update

[PATCH 2/2] block: assert that permission commit sets same permissions

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
On permission update commit we must set same permissions as on _check_. Let's add assertions. Next step may be to drop permission parameters from _set_. Note that prior to previous commit, fixing bdrv_drop_intermediate(), new assertion in bdrv_child_set_perm() crashes on iotests 30 and 40.

[PATCH RFC 00/21] block: update graph permissions update

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
Hi all! Here is a proposal of updating graph changing procedures. The thing brought me here is a question about "activating" filters after insertion, which is done in mirror_top and backup_top. The problem is that we can't simply avoid permission conflict when inserting the filter: during

Re: [PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Maxim Levitsky
On Mon, 2020-11-23 at 19:20 +0100, Alberto Garcia wrote: > On Mon 23 Nov 2020 05:09:41 PM CET, Kevin Wolf wrote: > > > Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") > > > introduced a subtle change to code in zero_in_l2_slice: > > > > > > It swapped the order of > > >

Re: [PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Alberto Garcia
On Mon 23 Nov 2020 05:09:41 PM CET, Kevin Wolf wrote: >> Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") >> introduced a subtle change to code in zero_in_l2_slice: >> >> It swapped the order of >> >> 1. qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); >> 2.

Re: [PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Maxim Levitsky
On Mon, 2020-11-23 at 18:38 +0100, Kevin Wolf wrote: > Am 23.11.2020 um 16:49 hat Maxim Levitsky geschrieben: > > Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") > > introduced a subtle change to code in zero_in_l2_slice: > > > > It swapped the order of > > > > 1.

Re: [PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Kevin Wolf
Am 23.11.2020 um 16:49 hat Maxim Levitsky geschrieben: > Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") > introduced a subtle change to code in zero_in_l2_slice: > > It swapped the order of > > 1. qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); > 2.

Re: [PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich
On 23.11.2020 18:44, Andrey Shinkevich wrote: The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from the

Re: [PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich
On 23.11.2020 18:44, Andrey Shinkevich wrote: The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from the

Re: [PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Kevin Wolf
Am 23.11.2020 um 16:49 hat Maxim Levitsky geschrieben: > Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") > introduced a subtle change to code in zero_in_l2_slice: > > It swapped the order of > > 1. qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); > 2.

[PATCH 0/1] Fix qcow2 corruption after addition of subcluster support

2020-11-23 Thread Maxim Levitsky
On this weekend, I had discovered that one of my VMs started to act weird. Due to this, I found out that it and most of the other VMs I have, have grown an qcow2 corruption. So after some bisecting, digging through dumps, and debugging, I think I found the root cause and a fix. In addition to

[PATCH 1/1] Fix qcow2 corruption on discard

2020-11-23 Thread Maxim Levitsky
Commit 205fa50750 ("qcow2: Add subcluster support to zero_in_l2_slice()") introduced a subtle change to code in zero_in_l2_slice: It swapped the order of 1. qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_slice); 2. set_l2_entry(s, l2_slice, l2_index + i, QCOW_OFLAG_ZERO); 3.

[PATCH v2 1/2] iotests: add another bash sleep command to 247

2020-11-23 Thread Andrey Shinkevich via
This patch paves the way for the one that follows. The following patch makes the QMP monitor to read up to 4K from stdin at once. That results in running the bash 'sleep' command before the _qemu_proc_exec() starts in subshell. Another 'sleep' command with an unobtrusive 'query-status' plays as a

[PATCH v2 2/2] monitor: increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse. This patch increases the amount of read data up to 4096 bytes that fits the buffer size on the channel level. A

[PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from the input buffer so that incoming requests do not

Re: [PULL 00/33] Block patches

2020-11-23 Thread Peter Maydell
On Mon, 23 Nov 2020 at 12:55, Philippe Mathieu-Daudé wrote: > > On 11/4/20 9:59 PM, Peter Maydell wrote: > > On Wed, 4 Nov 2020 at 15:18, Stefan Hajnoczi wrote: > >> > >> The following changes since commit > >> 8507c9d5c9a62de2a0e281b640f995e26eac46af: > >> > >> Merge remote-tracking branch

Re: [PATCH RFC 0/5] Fix accidental crash in iotest 30

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
23.11.2020 14:10, Kevin Wolf wrote: Am 23.11.2020 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: 23.11.2020 13:10, Kevin Wolf wrote: Am 20.11.2020 um 19:19 hat Vladimir Sementsov-Ogievskiy geschrieben: 20.11.2020 20:22, Kevin Wolf wrote: Am 20.11.2020 um 17:43 hat Vladimir

Re: [PULL 00/33] Block patches

2020-11-23 Thread Philippe Mathieu-Daudé
On 11/4/20 9:59 PM, Peter Maydell wrote: > On Wed, 4 Nov 2020 at 15:18, Stefan Hajnoczi wrote: >> >> The following changes since commit 8507c9d5c9a62de2a0e281b640f995e26eac46af: >> >> Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into >> staging (2020-11-03 15:59:44 +) >>

Re: [raw] Guest stuck during live live-migration

2020-11-23 Thread Kevin Wolf
[ Cc: qemu-block ] Am 23.11.2020 um 10:36 hat Quentin Grolleau geschrieben: > Hello, > > In our company, we are hosting a large number of Vm, hosted behind Openstack > (so libvirt/qemu). > A large majority of our Vms are runnign with local data only, stored on NVME, > and most of them are RAW

Re: [PATCH 0/4] vhost-user: avoid g_return_val_if() in get/set_config()

2020-11-23 Thread Stefano Garzarella
On Wed, Nov 18, 2020 at 09:16:40AM +, Stefan Hajnoczi wrote: Markus Armbruster pointed out that g_return_val_if() is meant for programming errors. It must not be used for input validation since it can be compiled out. Use explicit if statements instead. This patch series converts vhost-user

Re: [PATCH RFC 0/5] Fix accidental crash in iotest 30

2020-11-23 Thread Kevin Wolf
Am 23.11.2020 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: > 23.11.2020 13:10, Kevin Wolf wrote: > > Am 20.11.2020 um 19:19 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > 20.11.2020 20:22, Kevin Wolf wrote: > > > > Am 20.11.2020 um 17:43 hat Vladimir Sementsov-Ogievskiy geschrieben:

Re: [PATCH RFC 0/5] Fix accidental crash in iotest 30

2020-11-23 Thread Vladimir Sementsov-Ogievskiy
23.11.2020 13:10, Kevin Wolf wrote: Am 20.11.2020 um 19:19 hat Vladimir Sementsov-Ogievskiy geschrieben: 20.11.2020 20:22, Kevin Wolf wrote: Am 20.11.2020 um 17:43 hat Vladimir Sementsov-Ogievskiy geschrieben: 20.11.2020 19:36, Kevin Wolf wrote: Am 20.11.2020 um 17:16 hat Vladimir

Re: [PATCH RFC 0/5] Fix accidental crash in iotest 30

2020-11-23 Thread Kevin Wolf
Am 20.11.2020 um 19:19 hat Vladimir Sementsov-Ogievskiy geschrieben: > 20.11.2020 20:22, Kevin Wolf wrote: > > Am 20.11.2020 um 17:43 hat Vladimir Sementsov-Ogievskiy geschrieben: > > > 20.11.2020 19:36, Kevin Wolf wrote: > > > > Am 20.11.2020 um 17:16 hat Vladimir Sementsov-Ogievskiy geschrieben: