ImageInfo oddities regarding compression

[Markus Armbruster]

ImageInfo has an optional member @compressed:

##
# @ImageInfo:
[...]
# @compressed: true if the image is compressed (Since 1.7)

Doc bug: neglects to specify the default.  I guess it's false.

The only user appears to be vmdk_get_extent_info().  Goes back to
v1.7.0's commits

f4c129a38a vmdk: Implment bdrv_get_specific_info
cbe82d7fb3 qapi: Add optional field 'compressed' to ImageInfo

ImageInfo also has an optional member @format-specific.

Doc bug: neglects to specify when it's present.  I assume it's always
present when member @format has a value that has a non-empty variant in
@format-specific's type ImageInfoSpecific.

Format qcow2's variant is ImageInfoSpecificQCow2.  It has a mandatory
member @compression-type.

   ##
   # @Qcow2CompressionType:
   #
   # Compression type used in qcow2 image file
   #
   # @zlib: zlib compression, see 
   # @zstd: zstd compression, see 
   #
   # Since: 5.1
   ##
   { 'enum': 'Qcow2CompressionType',
 'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }

Apparently, you can't have a qcow2 image without compression.  Correct?

Can you imagine a use case for "without compression"?

I fell down this (thankfully shallow) rabbit hole because we also have

{ 'enum': 'MultiFDCompression',
  'data': [ 'none', 'zlib',
{ 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }

I wonder whether we could merge them into a common type.

Re: [PATCH v2 6/6] Rename arch_init.h to arch_type.h

[Cornelia Huck]

On Wed, 25 Nov 2020 15:56:36 -0500
Eduardo Habkost  wrote:

> The only declarations in arch_init.h are related to the arch_type
> variable (which is a useful feature that allows us to simplify
> command line option handling).  Rename the header to reflect its
> purpose.
> 
> Signed-off-by: Eduardo Habkost 
> ---
> Cc: Markus Armbruster 
> Cc: Kevin Wolf 
> Cc: Max Reitz 
> Cc: Paolo Bonzini 
> Cc: "Daniel P. Berrangé" 
> Cc: Eduardo Habkost 
> Cc: qemu-block@nongnu.org
> Cc: qemu-de...@nongnu.org
> ---
>  include/sysemu/{arch_init.h => arch_type.h} | 0
>  blockdev.c  | 2 +-
>  softmmu/arch_init.c | 2 +-
>  softmmu/qdev-monitor.c  | 2 +-
>  softmmu/vl.c| 2 +-
>  stubs/arch_type.c   | 2 +-
>  6 files changed, 5 insertions(+), 5 deletions(-)
>  rename include/sysemu/{arch_init.h => arch_type.h} (100%)

Reviewed-by: Cornelia Huck 

Re: ImageInfo oddities regarding compression

[Daniel P . Berrangé]

On Fri, Nov 27, 2020 at 11:06:36AM +0100, Markus Armbruster wrote:
> ImageInfo has an optional member @compressed:
> 
> ##
> # @ImageInfo:
> [...]
> # @compressed: true if the image is compressed (Since 1.7)
> 
> Doc bug: neglects to specify the default.  I guess it's false.
> 
> The only user appears to be vmdk_get_extent_info().  Goes back to
> v1.7.0's commits
> 
> f4c129a38a vmdk: Implment bdrv_get_specific_info
> cbe82d7fb3 qapi: Add optional field 'compressed' to ImageInfo
> 
> ImageInfo also has an optional member @format-specific.
> 
> Doc bug: neglects to specify when it's present.  I assume it's always
> present when member @format has a value that has a non-empty variant in
> @format-specific's type ImageInfoSpecific.
> 
> Format qcow2's variant is ImageInfoSpecificQCow2.  It has a mandatory
> member @compression-type.
> 
>##
># @Qcow2CompressionType:
>#
># Compression type used in qcow2 image file
>#
># @zlib: zlib compression, see 
># @zstd: zstd compression, see 
>#
># Since: 5.1
>##
>{ 'enum': 'Qcow2CompressionType',
>  'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
> 
> Apparently, you can't have a qcow2 image without compression.  Correct?
> 
> Can you imagine a use case for "without compression"?

Yes & no. An image always has a compression type, either implicitly
zlib as the historical default, or explicitly as a type recorded in
the header.  This doesn't mean compression is used.

There may be zero or more clusters actually using compression.
Typically compression will never be used, but there's still an
associated compression type regardless.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: ImageInfo oddities regarding compression

[Markus Armbruster]

Kevin Wolf  writes:

> Am 27.11.2020 um 11:14 hat Daniel P. Berrangé geschrieben:
>> On Fri, Nov 27, 2020 at 11:06:36AM +0100, Markus Armbruster wrote:
>> > ImageInfo has an optional member @compressed:
>> > 
>> > ##
>> > # @ImageInfo:
>> > [...]
>> > # @compressed: true if the image is compressed (Since 1.7)
>> > 
>> > Doc bug: neglects to specify the default.  I guess it's false.
>
> I think rather than false (definitely not compressed) it might just mean
> unknown, see below.
>
>> > The only user appears to be vmdk_get_extent_info().  Goes back to
>> > v1.7.0's commits
>> > 
>> > f4c129a38a vmdk: Implment bdrv_get_specific_info
>> > cbe82d7fb3 qapi: Add optional field 'compressed' to ImageInfo

Should it be in ImageInfoSpecificVmdk instead?

I know we can't just move it, and copy, deprecate, wait, delete may not
be worth the bother.  Sill, I'm curious :)

>> > ImageInfo also has an optional member @format-specific.
>> > 
>> > Doc bug: neglects to specify when it's present.  I assume it's always
>> > present when member @format has a value that has a non-empty variant in
>> > @format-specific's type ImageInfoSpecific.
>
> This seems to be the intention, yes.
>
>> > Format qcow2's variant is ImageInfoSpecificQCow2.  It has a mandatory
>> > member @compression-type.
>> > 
>> >##
>> ># @Qcow2CompressionType:
>> >#
>> ># Compression type used in qcow2 image file
>> >#
>> ># @zlib: zlib compression, see 
>> ># @zstd: zstd compression, see 
>> >#
>> ># Since: 5.1
>> >##
>> >{ 'enum': 'Qcow2CompressionType',
>> >  'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
>> > 
>> > Apparently, you can't have a qcow2 image without compression.  Correct?
>> > 
>> > Can you imagine a use case for "without compression"?
>> 
>> Yes & no. An image always has a compression type, either implicitly
>> zlib as the historical default, or explicitly as a type recorded in
>> the header.  This doesn't mean compression is used.
>> 
>> There may be zero or more clusters actually using compression.
>> Typically compression will never be used, but there's still an
>> associated compression type regardless.
>
> Right, so the correct answer to "is this image compressed?" is "unknown"
> for qcow2. Providing an answer would require checking all clusters in
> the image for the compression flag, which is not something we want to do
> for query-block. And even if we did that, it would still be unclear what
> to do with a partially compressed image.
>
> The @compression-type only tells you what format a compressed cluster
> uses if any compressed cluster exists in the image (or if a new
> compressed cluster is written to it). It doesn't mean that such clusters
> currently exist.

Makes sense.

If we had a compression type 'none', it would effectively tell us that
the image is definitely not compressed, because any "compressed"
clusters would use method none, i.e. be not compressed.  Observation,
not feature request.

>> I fell down this (thankfully shallow) rabbit hole because we also have
>> 
>> { 'enum': 'MultiFDCompression',
>>   'data': [ 'none', 'zlib',
>> { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
>> 
>> I wonder whether we could merge them into a common type.

Looks like we could: current code would never report the additional
value 'none'.  Introspection would show it, though.  Seems unlikely to
cause trouble.  Observation, not demand.

Re: ImageInfo oddities regarding compression

[Kevin Wolf]

Am 27.11.2020 um 11:14 hat Daniel P. Berrangé geschrieben:
> On Fri, Nov 27, 2020 at 11:06:36AM +0100, Markus Armbruster wrote:
> > ImageInfo has an optional member @compressed:
> > 
> > ##
> > # @ImageInfo:
> > [...]
> > # @compressed: true if the image is compressed (Since 1.7)
> > 
> > Doc bug: neglects to specify the default.  I guess it's false.

I think rather than false (definitely not compressed) it might just mean
unknown, see below.

> > The only user appears to be vmdk_get_extent_info().  Goes back to
> > v1.7.0's commits
> > 
> > f4c129a38a vmdk: Implment bdrv_get_specific_info
> > cbe82d7fb3 qapi: Add optional field 'compressed' to ImageInfo
> > 
> > ImageInfo also has an optional member @format-specific.
> > 
> > Doc bug: neglects to specify when it's present.  I assume it's always
> > present when member @format has a value that has a non-empty variant in
> > @format-specific's type ImageInfoSpecific.

This seems to be the intention, yes.

> > Format qcow2's variant is ImageInfoSpecificQCow2.  It has a mandatory
> > member @compression-type.
> > 
> >##
> ># @Qcow2CompressionType:
> >#
> ># Compression type used in qcow2 image file
> >#
> ># @zlib: zlib compression, see 
> ># @zstd: zstd compression, see 
> >#
> ># Since: 5.1
> >##
> >{ 'enum': 'Qcow2CompressionType',
> >  'data': [ 'zlib', { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
> > 
> > Apparently, you can't have a qcow2 image without compression.  Correct?
> > 
> > Can you imagine a use case for "without compression"?
> 
> Yes & no. An image always has a compression type, either implicitly
> zlib as the historical default, or explicitly as a type recorded in
> the header.  This doesn't mean compression is used.
> 
> There may be zero or more clusters actually using compression.
> Typically compression will never be used, but there's still an
> associated compression type regardless.

Right, so the correct answer to "is this image compressed?" is "unknown"
for qcow2. Providing an answer would require checking all clusters in
the image for the compression flag, which is not something we want to do
for query-block. And even if we did that, it would still be unclear what
to do with a partially compressed image.

The @compression-type only tells you what format a compressed cluster
uses if any compressed cluster exists in the image (or if a new
compressed cluster is written to it). It doesn't mean that such clusters
currently exist.

Kevin

[PATCH v3 0/5] Increase amount of data for monitor to read

[Andrey Shinkevich via]

The subject was discussed here:
https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html
https://patchew.org/QEMU/20190610105906.28524-1-dplotni...@virtuozzo.com/#
Message-ID: <31dd78ba-bd64-2ed6-3c8f-eed4e904d...@virtuozzo.com>
and v2:
Message-Id: <1606146274-246154-1-git-send-email-andrey.shinkev...@virtuozzo.com>

This series is a solution for the issue with overflow of the monitor queue
with QMP requests if we keep the maximum queue length unchanged (=8).

v3:
  01: New
  02: New
  03: The additional little JSON parser removed and the resources of the
  existing JSON parser were used to track the end of a QMP command.
  04: The amount of read input data increases only.

Andrey Shinkevich (4):
  monitor: change function obsolete name in comments
  monitor: drain requests queue with 'channel closed' event
  monitor: let QMP monitor track JSON message content
  monitor: increase amount of data for monitor to read

Vladimir Sementsov-Ogievskiy (1):
  iotests: 129 don't check backup "busy"

 include/qapi/qmp/json-parser.h |  5 ++--
 monitor/monitor.c  |  2 +-
 monitor/qmp.c  | 66 --
 qga/main.c |  2 +-
 qobject/json-lexer.c   | 30 +--
 qobject/json-parser-int.h  |  8 +++--
 qobject/json-streamer.c| 15 +-
 qobject/qjson.c|  2 +-
 tests/qemu-iotests/129 |  1 -
 tests/qtest/libqtest.c |  2 +-
 10 files changed, 79 insertions(+), 54 deletions(-)

-- 
1.8.3.1

[PATCH v3 4/5] iotests: 129 don't check backup "busy"

[Andrey Shinkevich via]

From: Vladimir Sementsov-Ogievskiy 

Busy is racy, job has it's "pause-points" when it's not busy. Drop this
check.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
Reviewed-by: Max Reitz 
---
 tests/qemu-iotests/129 | 1 -
 1 file changed, 1 deletion(-)

diff --git a/tests/qemu-iotests/129 b/tests/qemu-iotests/129
index 0e13244..3c22f64 100755
--- a/tests/qemu-iotests/129
+++ b/tests/qemu-iotests/129
@@ -67,7 +67,6 @@ class TestStopWithBlockJob(iotests.QMPTestCase):
 result = self.vm.qmp("stop")
 self.assert_qmp(result, 'return', {})
 result = self.vm.qmp("query-block-jobs")
-self.assert_qmp(result, 'return[0]/busy', True)
 self.assert_qmp(result, 'return[0]/ready', False)
 
 def test_drive_mirror(self):
-- 
1.8.3.1

[PATCH v3 3/5] monitor: let QMP monitor track JSON message content

[Andrey Shinkevich via]

We are going to allow the QMP monitor reading data from input channel
more than one byte at once to increase the performance. With the OOB
compatibility disabled, the monitor queues one QMP command at most. It
was done for the backward compatibility as stated in the comment before
pushing a command into the queue. To keep that concept functional, the
monitor should track the end of a single QMP command. It allows the
dispatcher handling the command and send a response to client in time.

Signed-off-by: Andrey Shinkevich 
---
 include/qapi/qmp/json-parser.h |  5 +++--
 monitor/qmp.c  | 18 --
 qga/main.c |  2 +-
 qobject/json-lexer.c   | 30 +-
 qobject/json-parser-int.h  |  8 +---
 qobject/json-streamer.c| 15 ---
 qobject/qjson.c|  2 +-
 tests/qtest/libqtest.c |  2 +-
 8 files changed, 56 insertions(+), 26 deletions(-)

diff --git a/include/qapi/qmp/json-parser.h b/include/qapi/qmp/json-parser.h
index 7345a9b..039addb 100644
--- a/include/qapi/qmp/json-parser.h
+++ b/include/qapi/qmp/json-parser.h
@@ -36,8 +36,9 @@ void json_message_parser_init(JSONMessageParser *parser,
Error *err),
   void *opaque, va_list *ap);
 
-void json_message_parser_feed(JSONMessageParser *parser,
- const char *buffer, size_t size);
+size_t  json_message_parser_feed(JSONMessageParser *parser,
+ const char *buffer, size_t size,
+ bool track_qmp);
 
 void json_message_parser_flush(JSONMessageParser *parser);
 
diff --git a/monitor/qmp.c b/monitor/qmp.c
index a86ed35..0b39c62 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -367,8 +367,22 @@ static void handle_qmp_command(void *opaque, QObject *req, 
Error *err)
 static void monitor_qmp_read(void *opaque, const uint8_t *buf, int size)
 {
 MonitorQMP *mon = opaque;
-
-json_message_parser_feed(>parser, (const char *) buf, size);
+char *cursor = (char *) buf;
+size_t len;
+
+while (size > 0) {
+len = json_message_parser_feed(>parser, (const char *) cursor,
+   size, true);
+cursor += len;
+size -= len;
+
+if (size > 0) {
+/* Let the dispatcher process the QMP command */
+while (qatomic_mb_read(>common.suspend_cnt)) {
+g_usleep(20);
+}
+}
+}
 }
 
 static QDict *qmp_greeting(MonitorQMP *mon)
diff --git a/qga/main.c b/qga/main.c
index dea6a3a..16de642 100644
--- a/qga/main.c
+++ b/qga/main.c
@@ -605,7 +605,7 @@ static gboolean channel_event_cb(GIOCondition condition, 
gpointer data)
 case G_IO_STATUS_NORMAL:
 buf[count] = 0;
 g_debug("read data, count: %d, data: %s", (int)count, buf);
-json_message_parser_feed(>parser, (char *)buf, (int)count);
+json_message_parser_feed(>parser, (char *)buf, (int)count, false);
 break;
 case G_IO_STATUS_EOF:
 g_debug("received EOF");
diff --git a/qobject/json-lexer.c b/qobject/json-lexer.c
index 632320d..1fefbae 100644
--- a/qobject/json-lexer.c
+++ b/qobject/json-lexer.c
@@ -280,10 +280,11 @@ void json_lexer_init(JSONLexer *lexer, bool 
enable_interpolation)
 lexer->x = lexer->y = 0;
 }
 
-static void json_lexer_feed_char(JSONLexer *lexer, char ch, bool flush)
+static JSONTokenType json_lexer_feed_char(JSONLexer *lexer, char ch, bool 
flush)
 {
 int new_state;
 bool char_consumed = false;
+JSONTokenType ret;
 
 lexer->x++;
 if (ch == '\n') {
@@ -310,16 +311,16 @@ static void json_lexer_feed_char(JSONLexer *lexer, char 
ch, bool flush)
 case JSON_FLOAT:
 case JSON_KEYWORD:
 case JSON_STRING:
-json_message_process_token(lexer, lexer->token, new_state,
-   lexer->x, lexer->y);
+ret = json_message_process_token(lexer, lexer->token, new_state,
+ lexer->x, lexer->y);
 /* fall through */
 case IN_START:
 g_string_truncate(lexer->token, 0);
 new_state = lexer->start_state;
 break;
 case JSON_ERROR:
-json_message_process_token(lexer, lexer->token, JSON_ERROR,
-   lexer->x, lexer->y);
+ret = json_message_process_token(lexer, lexer->token, JSON_ERROR,
+ lexer->x, lexer->y);
 new_state = IN_RECOVERY;
 /* fall through */
 case IN_RECOVERY:
@@ -335,20 +336,31 @@ static void json_lexer_feed_char(JSONLexer *lexer, char 
ch, bool flush)
  * this is a security consideration.
  */
 if (lexer->token->len > MAX_TOKEN_SIZE) {
-json_message_process_token(lexer, lexer->token, lexer->state,
-  

[PATCH v2 03/36] block: bdrv_append(): don't consume reference

[Vladimir Sementsov-Ogievskiy]

We have too much comments for this feature. It seems better just don't
do it. Most of real users (tests don't count) have to create additional
reference.

Drop also comment in external_snapshot_prepare:
 - bdrv_append doesn't "remove" old bs in common sense, it sounds
   strange
 - the fact that bdrv_append can fail is obvious from the context
 - the fact that we must rollback all changes in transaction abort is
   known (it's the direct role of abort)

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 19 +++
 block/backup-top.c  |  1 -
 block/commit.c  |  1 +
 block/mirror.c  |  3 ---
 blockdev.c  |  4 
 tests/test-bdrv-drain.c |  2 +-
 tests/test-bdrv-graph-mod.c |  2 ++
 7 files changed, 7 insertions(+), 25 deletions(-)

diff --git a/block.c b/block.c
index 0dd28f0902..55efef3c9d 100644
--- a/block.c
+++ b/block.c
@@ -3145,11 +3145,6 @@ static BlockDriverState 
*bdrv_append_temp_snapshot(BlockDriverState *bs,
 goto out;
 }
 
-/* bdrv_append() consumes a strong reference to bs_snapshot
- * (i.e. it will call bdrv_unref() on it) even on error, so in
- * order to be able to return one, we have to increase
- * bs_snapshot's refcount here */
-bdrv_ref(bs_snapshot);
 bdrv_append(bs_snapshot, bs, _err);
 if (local_err) {
 error_propagate(errp, local_err);
@@ -4608,10 +4603,8 @@ void bdrv_replace_node(BlockDriverState *from, 
BlockDriverState *to,
  *
  * This function does not create any image files.
  *
- * bdrv_append() takes ownership of a bs_new reference and unrefs it because
- * that's what the callers commonly need. bs_new will be referenced by the old
- * parents of bs_top after bdrv_append() returns. If the caller needs to keep a
- * reference of its own, it must call bdrv_ref().
+ * Recent update: bdrv_append does NOT eat bs_new reference for now. Drop this
+ * comment several moths later.
  */
 void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
  Error **errp)
@@ -4621,20 +4614,14 @@ void bdrv_append(BlockDriverState *bs_new, 
BlockDriverState *bs_top,
 bdrv_set_backing_hd(bs_new, bs_top, _err);
 if (local_err) {
 error_propagate(errp, local_err);
-goto out;
+return;
 }
 
 bdrv_replace_node(bs_top, bs_new, _err);
 if (local_err) {
 error_propagate(errp, local_err);
 bdrv_set_backing_hd(bs_new, NULL, _abort);
-goto out;
 }
-
-/* bs_new is now referenced by its new parents, we don't need the
- * additional reference any more. */
-out:
-bdrv_unref(bs_new);
 }
 
 static void bdrv_delete(BlockDriverState *bs)
diff --git a/block/backup-top.c b/block/backup-top.c
index fe6883cc97..650ed6195c 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -222,7 +222,6 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState 
*source,
 
 bdrv_drained_begin(source);
 
-bdrv_ref(top);
 bdrv_append(top, source, _err);
 if (local_err) {
 error_prepend(_err, "Cannot append backup-top filter: ");
diff --git a/block/commit.c b/block/commit.c
index 71db7ba747..61924bcf66 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -313,6 +313,7 @@ void commit_start(const char *job_id, BlockDriverState *bs,
 commit_top_bs->total_sectors = top->total_sectors;
 
 bdrv_append(commit_top_bs, top, _err);
+bdrv_unref(commit_top_bs); /* referenced by new parents or failed */
 if (local_err) {
 commit_top_bs = NULL;
 error_propagate(errp, local_err);
diff --git a/block/mirror.c b/block/mirror.c
index 8e1ad6eceb..13f7ecc998 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1605,9 +1605,6 @@ static BlockJob *mirror_start_job(
 bs_opaque = g_new0(MirrorBDSOpaque, 1);
 mirror_top_bs->opaque = bs_opaque;
 
-/* bdrv_append takes ownership of the mirror_top_bs reference, need to keep
- * it alive until block_job_create() succeeds even if bs has no parent. */
-bdrv_ref(mirror_top_bs);
 bdrv_drained_begin(bs);
 bdrv_append(mirror_top_bs, bs, _err);
 bdrv_drained_end(bs);
diff --git a/blockdev.c b/blockdev.c
index b5f11c524b..96c96f8ba6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1587,10 +1587,6 @@ static void external_snapshot_prepare(BlkActionState 
*common,
 goto out;
 }
 
-/* This removes our old bs and adds the new bs. This is an operation that
- * can fail, so we need to do it in .prepare; undoing it for abort is
- * always possible. */
-bdrv_ref(state->new_bs);
 bdrv_append(state->new_bs, state->old_bs, _err);
 if (local_err) {
 error_propagate(errp, local_err);
diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c
index 8a29e33e00..892f7f47d8 100644
--- a/tests/test-bdrv-drain.c
+++ b/tests/test-bdrv-drain.c
@@ -1478,7 +1478,6 @@ static void test_append_to_drained(void)
 g_assert_cmpint(base_s->drain_count, ==, 1);
 

[PATCH v2 25/36] block: introduce bdrv_drop_filter()

[Vladimir Sementsov-Ogievskiy]

Using bdrv_replace_node() for removing filter is not good enough: it
keeps child reference of the filter, which may conflict with original
top node during permission update.

Instead let's create new interface, which will do all graph
modifications first and then update permissions.

Let's modify bdrv_replace_node_common(), allowing it additionally drop
backing chain child link pointing to new node. This is quite
appropriate for bdrv_drop_intermediate() and makes possible to add
new bdrv_drop_filter() as a simple wrapper.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |  1 +
 block.c   | 42 ++
 2 files changed, 39 insertions(+), 4 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 8f6100dad7..0f21ef313f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -348,6 +348,7 @@ int bdrv_append(BlockDriverState *bs_new, BlockDriverState 
*bs_top,
 Error **errp);
 int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
   Error **errp);
+int bdrv_drop_filter(BlockDriverState *bs, Error **errp);
 
 int bdrv_parse_aio(const char *mode, int *flags);
 int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
diff --git a/block.c b/block.c
index b1394b721c..e835a78f06 100644
--- a/block.c
+++ b/block.c
@@ -4919,7 +4919,6 @@ static TransactionActionDrv bdrv_remove_backing_drv = {
 .commit = bdrv_child_free,
 };
 
-__attribute__((unused))
 static void bdrv_remove_backing(BlockDriverState *bs, GSList **tran)
 {
 if (!bs->backing) {
@@ -4968,15 +4967,30 @@ static int bdrv_replace_node_noperm(BlockDriverState 
*from,
  *
  * With auto_skip=false the error is returned if from has a parent which should
  * not be updated.
+ *
+ * With detach_subchain to must be in a backing chain of from. In this case
+ * backing link of the cow-parent of @to is removed.
  */
 static int bdrv_replace_node_common(BlockDriverState *from,
 BlockDriverState *to,
-bool auto_skip, Error **errp)
+bool auto_skip, bool detach_subchain,
+Error **errp)
 {
 int ret = -EPERM;
 GSList *tran = NULL;
 g_autoptr(GHashTable) found = NULL;
 g_autoptr(GSList) refresh_list = NULL;
+BlockDriverState *to_cow_parent;
+
+if (detach_subchain) {
+assert(bdrv_chain_contains(from, to));
+for (to_cow_parent = from;
+ bdrv_filter_or_cow_bs(to_cow_parent) != to;
+ to_cow_parent = bdrv_filter_or_cow_bs(to_cow_parent))
+{
+;
+}
+}
 
 /* Make sure that @from doesn't go away until we have successfully attached
  * all of its parents to @to. */
@@ -4997,6 +5011,10 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
 goto out;
 }
 
+if (detach_subchain) {
+bdrv_remove_backing(to_cow_parent, );
+}
+
 found = g_hash_table_new(NULL, NULL);
 
 refresh_list = bdrv_topological_dfs(refresh_list, found, to);
@@ -5016,7 +5034,13 @@ out:
 int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
   Error **errp)
 {
-return bdrv_replace_node_common(from, to, true, errp);
+return bdrv_replace_node_common(from, to, true, false, errp);
+}
+
+int bdrv_drop_filter(BlockDriverState *bs, Error **errp)
+{
+return bdrv_replace_node_common(bs, bdrv_filter_or_cow_bs(bs), true, true,
+errp);
 }
 
 /*
@@ -5326,7 +5350,17 @@ int bdrv_drop_intermediate(BlockDriverState *top, 
BlockDriverState *base,
 updated_children = g_slist_prepend(updated_children, c);
 }
 
-bdrv_replace_node_common(top, base, false, _err);
+/*
+ * It seems correct to pass detach_subchain=true here, but it triggers
+ * one more yet not fixed bug, when due to nested aio_poll loop we switch 
to
+ * another drained section, which modify the graph (for example, removing
+ * the child, which we keep in updated_children list). So, it's a TODO.
+ *
+ * Note, bug triggered if pass detach_subchain=true here and run
+ * test-bdrv-drain. test_drop_intermediate_poll() test-case will crash.
+ * That's a FIXME.
+ */
+bdrv_replace_node_common(top, base, false, false, _err);
 if (local_err) {
 error_report_err(local_err);
 goto exit;
-- 
2.21.3

Re: [PATCH v2 2/2] monitor: increase amount of data for monitor to read

[Andrey Shinkevich]

On 24.11.2020 14:03, Vladimir Sementsov-Ogievskiy wrote:

23.11.2020 18:44, Andrey Shinkevich wrote:

QMP and HMP monitors read one byte at a time from the socket or stdin,
which is very inefficient. With 100+ VMs on the host, this results in
multiple extra system calls and CPU overuse.
This patch increases the amount of read data up to 4096 bytes that fits
the buffer size on the channel level.
A JSON little parser is introduced to throttle QMP commands read from
the buffer so that incoming requests do not overflow the monitor input
queue.

Suggested-by: Denis V. Lunev
Signed-off-by: Andrey Shinkevich



Can't we just increase qmp queue instead? It seems a lot simpler:



With the OOB compatibility disabled, the monitor queues one QMP command 
at most. It was made for the backward compatibility as stated in the 
comment before pushing a command into the queue. To keep that concept 
functional, the monitor should track the end of a single QMP command. It 
allows the dispatcher handling the command and send a response to client 
in time.
With the patch below, the monitor queue will be filled with QMP commands 
as many as they will be found in the input buffer. The first command 
execution {"execute":"qmp_capabilities"} takes more time and queue will 
be filled at full. Then the dispatcher starts execution of other 
commands in the monitor queue. The process becomes synchronious. In this 
case, we need neither thread nor the queue.


Andrey



diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index 348bfad3d5..7e721eee3f 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -8,7 +8,7 @@
  typedef struct MonitorHMP MonitorHMP;
  typedef struct MonitorOptions MonitorOptions;

-#define QMP_REQ_QUEUE_LEN_MAX 8
+#define QMP_REQ_QUEUE_LEN_MAX 4096

  extern QemuOptsList qemu_mon_opts;

diff --git a/monitor/monitor.c b/monitor/monitor.c
index 84222cd130..1588f00306 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -566,7 +566,7 @@ int monitor_can_read(void *opaque)
  {
  Monitor *mon = opaque;

-    return !qatomic_mb_read(>suspend_cnt);
+    return !qatomic_mb_read(>suspend_cnt) ? 4096 : 0;
  }


- with this patch tests pass and performance is even better.

[PATCH v3 2/5] monitor: drain requests queue with 'channel closed' event

[Andrey Shinkevich via]

When CHR_EVENT_CLOSED comes, the QMP requests queue may still contain
unprocessed commands. It can happen with QMP capability OOB enabled.
Let the dispatcher complete handling requests rest in the monitor
queue.

Signed-off-by: Andrey Shinkevich 
---
 monitor/qmp.c | 46 +-
 1 file changed, 21 insertions(+), 25 deletions(-)

diff --git a/monitor/qmp.c b/monitor/qmp.c
index 7169366..a86ed35 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -75,36 +75,32 @@ static void monitor_qmp_cleanup_req_queue_locked(MonitorQMP 
*mon)
 }
 }
 
-static void monitor_qmp_cleanup_queue_and_resume(MonitorQMP *mon)
+/*
+ * Let unprocessed QMP commands be handled.
+ */
+static void monitor_qmp_drain_queue(MonitorQMP *mon)
 {
-qemu_mutex_lock(>qmp_queue_lock);
+bool q_is_empty = false;
 
-/*
- * Same condition as in monitor_qmp_dispatcher_co(), but before
- * removing an element from the queue (hence no `- 1`).
- * Also, the queue should not be empty either, otherwise the
- * monitor hasn't been suspended yet (or was already resumed).
- */
-bool need_resume = (!qmp_oob_enabled(mon) ||
-mon->qmp_requests->length == QMP_REQ_QUEUE_LEN_MAX)
-&& !g_queue_is_empty(mon->qmp_requests);
+while (!q_is_empty) {
+qemu_mutex_lock(>qmp_queue_lock);
+q_is_empty = g_queue_is_empty(mon->qmp_requests);
+qemu_mutex_unlock(>qmp_queue_lock);
 
-monitor_qmp_cleanup_req_queue_locked(mon);
+if (!q_is_empty) {
+if (!qatomic_xchg(_dispatcher_co_busy, true)) {
+/* Kick the dispatcher coroutine */
+aio_co_wake(qmp_dispatcher_co);
+} else {
+/* Let the dispatcher do its job for a while */
+g_usleep(40);
+}
+}
+}
 
-if (need_resume) {
-/*
- * handle_qmp_command() suspended the monitor because the
- * request queue filled up, to be resumed when the queue has
- * space again.  We just emptied it; resume the monitor.
- *
- * Without this, the monitor would remain suspended forever
- * when we get here while the monitor is suspended.  An
- * unfortunately timed CHR_EVENT_CLOSED can do the trick.
- */
+if (qatomic_mb_read(>common.suspend_cnt)) {
 monitor_resume(>common);
 }
-
-qemu_mutex_unlock(>qmp_queue_lock);
 }
 
 void qmp_send_response(MonitorQMP *mon, const QDict *rsp)
@@ -418,7 +414,7 @@ static void monitor_qmp_event(void *opaque, QEMUChrEvent 
event)
  * stdio, it's possible that stdout is still open when stdin
  * is closed.
  */
-monitor_qmp_cleanup_queue_and_resume(mon);
+monitor_qmp_drain_queue(mon);
 json_message_parser_destroy(>parser);
 json_message_parser_init(>parser, handle_qmp_command,
  mon, NULL);
-- 
1.8.3.1

[PATCH v3 5/5] monitor: increase amount of data for monitor to read

[Andrey Shinkevich via]

QMP and HMP monitors read one byte at a time from the socket or stdin,
which is very inefficient. With 100+ VMs on the host, this results in
multiple extra system calls and CPU overuse.
This patch increases the amount of read data up to 4096 bytes that fits
the buffer size on the channel level.

Suggested-by: Denis V. Lunev 
Signed-off-by: Andrey Shinkevich 
---
 monitor/monitor.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/monitor/monitor.c b/monitor/monitor.c
index 84222cd..43d2d3b 100644
--- a/monitor/monitor.c
+++ b/monitor/monitor.c
@@ -566,7 +566,7 @@ int monitor_can_read(void *opaque)
 {
 Monitor *mon = opaque;
 
-return !qatomic_mb_read(>suspend_cnt);
+return !qatomic_mb_read(>suspend_cnt) ? CHR_READ_BUF_LEN : 0;
 }
 
 void monitor_list_append(Monitor *mon)
-- 
1.8.3.1

[PATCH v2 20/36] block: add bdrv_attach_child_common() transaction action

[Vladimir Sementsov-Ogievskiy]

Split out no-perm part of bdrv_root_attach_child() into separate
transaction action. bdrv_root_attach_child() now moves to new
permission update paradigm: first update graph relations then update
permissions.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 162 
 1 file changed, 117 insertions(+), 45 deletions(-)

diff --git a/block.c b/block.c
index f0fcd7..a7ccbb4fb1 100644
--- a/block.c
+++ b/block.c
@@ -86,6 +86,13 @@ static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, 
AioContext *ctx,
GSList **ignore);
 static void bdrv_replace_child_noperm(BdrvChild *child,
   BlockDriverState *new_bs);
+static int bdrv_attach_child_common(BlockDriverState *child_bs,
+const char *child_name,
+const BdrvChildClass *child_class,
+BdrvChildRole child_role,
+uint64_t perm, uint64_t shared_perm,
+void *opaque, BdrvChild **child,
+GSList **tran, Error **errp);
 
 static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue
*queue, Error **errp);
@@ -2898,55 +2905,22 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
   uint64_t perm, uint64_t shared_perm,
   void *opaque, Error **errp)
 {
-BdrvChild *child;
-Error *local_err = NULL;
 int ret;
-AioContext *ctx;
+BdrvChild *child = NULL;
+GSList *tran = NULL;
 
-ret = bdrv_check_update_perm(child_bs, NULL, perm, shared_perm, NULL, 
errp);
+ret = bdrv_attach_child_common(child_bs, child_name, child_class,
+   child_role, perm, shared_perm, opaque,
+   , , errp);
 if (ret < 0) {
-bdrv_abort_perm_update(child_bs);
 bdrv_unref(child_bs);
 return NULL;
 }
 
-child = g_new(BdrvChild, 1);
-*child = (BdrvChild) {
-.bs = NULL,
-.name   = g_strdup(child_name),
-.klass  = child_class,
-.role   = child_role,
-.perm   = perm,
-.shared_perm= shared_perm,
-.opaque = opaque,
-};
-
-ctx = bdrv_child_get_parent_aio_context(child);
-
-/* If the AioContexts don't match, first try to move the subtree of
- * child_bs into the AioContext of the new parent. If this doesn't work,
- * try moving the parent into the AioContext of child_bs instead. */
-if (bdrv_get_aio_context(child_bs) != ctx) {
-ret = bdrv_try_set_aio_context(child_bs, ctx, _err);
-if (ret < 0) {
-if (bdrv_parent_try_set_aio_context(child, ctx, NULL) == 0) {
-ret = 0;
-error_free(local_err);
-local_err = NULL;
-}
-}
-if (ret < 0) {
-error_propagate(errp, local_err);
-g_free(child);
-bdrv_abort_perm_update(child_bs);
-bdrv_unref(child_bs);
-return NULL;
-}
-}
-
-/* This performs the matching bdrv_set_perm() for the above check. */
-bdrv_replace_child(child, child_bs);
+ret = bdrv_refresh_perms(child_bs, errp);
+tran_finalize(tran, ret);
 
+bdrv_unref(child_bs);
 return child;
 }
 
@@ -2988,16 +2962,114 @@ BdrvChild *bdrv_attach_child(BlockDriverState 
*parent_bs,
 return child;
 }
 
-static void bdrv_detach_child(BdrvChild *child)
+static void bdrv_remove_empty_child(BdrvChild *child)
 {
+assert(!child->bs);
 QLIST_SAFE_REMOVE(child, next);
-
-bdrv_replace_child(child, NULL);
-
 g_free(child->name);
 g_free(child);
 }
 
+typedef struct BdrvAttachChildCommonState {
+BdrvChild **child;
+AioContext *old_parent_ctx;
+AioContext *old_child_ctx;
+} BdrvAttachChildCommonState;
+
+static void bdrv_attach_child_common_abort(void *opaque)
+{
+BdrvAttachChildCommonState *s = opaque;
+BdrvChild *child = *s->child;
+BlockDriverState *bs = child->bs;
+
+bdrv_replace_child_noperm(child, NULL);
+
+if (bdrv_get_aio_context(bs) != s->old_child_ctx) {
+bdrv_try_set_aio_context(bs, s->old_child_ctx, _abort);
+}
+
+if (bdrv_child_get_parent_aio_context(child) != s->old_parent_ctx) {
+bdrv_parent_try_set_aio_context(child, s->old_parent_ctx,
+_abort);
+}
+
+bdrv_unref(bs);
+bdrv_remove_empty_child(child);
+*s->child = NULL;
+}
+
+static TransactionActionDrv bdrv_attach_child_common_drv = {
+.abort = bdrv_attach_child_common_abort,
+};
+
+/*
+ * Common part of attoching bdrv child to bs or to blk or to job
+ */
+static int 

[PATCH v2 21/36] block: add bdrv_attach_child_noperm() transaction action

[Vladimir Sementsov-Ogievskiy]

Split no-perm part of bdrv_attach_child as separate transaction action.
It will be used in later commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 71 ++---
 1 file changed, 58 insertions(+), 13 deletions(-)

diff --git a/block.c b/block.c
index a7ccbb4fb1..162a247579 100644
--- a/block.c
+++ b/block.c
@@ -86,6 +86,14 @@ static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, 
AioContext *ctx,
GSList **ignore);
 static void bdrv_replace_child_noperm(BdrvChild *child,
   BlockDriverState *new_bs);
+static int bdrv_attach_child_noperm(BlockDriverState *parent_bs,
+BlockDriverState *child_bs,
+const char *child_name,
+const BdrvChildClass *child_class,
+BdrvChildRole child_role,
+BdrvChild **child,
+GSList **tran,
+Error **errp);
 static int bdrv_attach_child_common(BlockDriverState *child_bs,
 const char *child_name,
 const BdrvChildClass *child_class,
@@ -2942,23 +2950,26 @@ BdrvChild *bdrv_attach_child(BlockDriverState 
*parent_bs,
  BdrvChildRole child_role,
  Error **errp)
 {
-BdrvChild *child;
-uint64_t perm, shared_perm;
-
-bdrv_get_cumulative_perm(parent_bs, , _perm);
+int ret;
+BdrvChild *child = NULL;
+GSList *tran = NULL;
 
-assert(parent_bs->drv);
-bdrv_child_perm(parent_bs, child_bs, NULL, child_role, NULL,
-perm, shared_perm, , _perm);
+ret = bdrv_attach_child_noperm(parent_bs, child_bs, child_name, 
child_class,
+   child_role, , , errp);
+if (ret < 0) {
+goto out;
+}
 
-child = bdrv_root_attach_child(child_bs, child_name, child_class,
-   child_role, perm, shared_perm, parent_bs,
-   errp);
-if (child == NULL) {
-return NULL;
+ret = bdrv_refresh_perms(parent_bs, errp);
+if (ret < 0) {
+goto out;
 }
 
-QLIST_INSERT_HEAD(_bs->children, child, next);
+out:
+tran_finalize(tran, ret);
+
+bdrv_unref(child_bs);
+
 return child;
 }
 
@@ -3064,6 +3075,40 @@ static int bdrv_attach_child_common(BlockDriverState 
*child_bs,
 return 0;
 }
 
+static int bdrv_attach_child_noperm(BlockDriverState *parent_bs,
+BlockDriverState *child_bs,
+const char *child_name,
+const BdrvChildClass *child_class,
+BdrvChildRole child_role,
+BdrvChild **child,
+GSList **tran,
+Error **errp)
+{
+int ret;
+uint64_t perm, shared_perm;
+
+assert(parent_bs->drv);
+
+bdrv_get_cumulative_perm(parent_bs, , _perm);
+bdrv_child_perm(parent_bs, child_bs, NULL, child_role, NULL,
+perm, shared_perm, , _perm);
+
+ret = bdrv_attach_child_common(child_bs, child_name, child_class,
+   child_role, perm, shared_perm, parent_bs,
+   child, tran, errp);
+if (ret < 0) {
+return ret;
+}
+
+QLIST_INSERT_HEAD(_bs->children, *child, next);
+/*
+ * child is removed in bdrv_attach_child_common_abort(), so don't care to
+ * abort this change separately.
+ */
+
+return 0;
+}
+
 static void bdrv_detach_child(BdrvChild *child)
 {
 bdrv_replace_child(child, NULL);
-- 
2.21.3

[PATCH v2 10/36] util: add transactions.c

[Vladimir Sementsov-Ogievskiy]

Add simple transaction API to use in further update of block graph
operations.

Supposed usage is:

- "prepare" is main function of the action and it should make the main
  effect of the action to be visible for the following actions, keeping
  possibility of roll-back, saving necessary things in action state,
  which is prepended to the list. So, driver struct doesn't include
  "prepare" field, as it is supposed to be called directly.

- commit/rollback is supposed to be called for the list of action
  states, to commit/rollback all the actions in reverse order

- When possible "commit" should not make visible effect for other
  actions, which make possible transparent logical interaction between
  actions.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/qemu/transactions.h | 46 +
 util/transactions.c | 81 +
 util/meson.build|  1 +
 3 files changed, 128 insertions(+)
 create mode 100644 include/qemu/transactions.h
 create mode 100644 util/transactions.c

diff --git a/include/qemu/transactions.h b/include/qemu/transactions.h
new file mode 100644
index 00..a5b15f46ab
--- /dev/null
+++ b/include/qemu/transactions.h
@@ -0,0 +1,46 @@
+/*
+ * Simple transactions API
+ *
+ * Copyright (c) 2020 Virtuozzo International GmbH.
+ *
+ * Author:
+ *  Vladimir Sementsov-Ogievskiy 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see .
+ */
+
+#ifndef QEMU_TRANSACTIONS_H
+#define QEMU_TRANSACTIONS_H
+
+#include 
+
+typedef struct TransactionActionDrv {
+void (*abort)(void *opeque);
+void (*commit)(void *opeque);
+void (*clean)(void *opeque);
+} TransactionActionDrv;
+
+void tran_prepend(GSList **list, TransactionActionDrv *drv, void *opaque);
+void tran_abort(GSList *backup);
+void tran_commit(GSList *backup);
+static inline void tran_finalize(GSList *backup, int ret)
+{
+if (ret < 0) {
+tran_abort(backup);
+} else {
+tran_commit(backup);
+}
+}
+
+#endif /* QEMU_TRANSACTIONS_H */
diff --git a/util/transactions.c b/util/transactions.c
new file mode 100644
index 00..ef1b9a36a4
--- /dev/null
+++ b/util/transactions.c
@@ -0,0 +1,81 @@
+/*
+ * Simple transactions API
+ *
+ * Copyright (c) 2020 Virtuozzo International GmbH.
+ *
+ * Author:
+ *  Sementsov-Ogievskiy Vladimir 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see .
+ */
+
+#include "qemu/osdep.h"
+
+#include "qemu/transactions.h"
+
+typedef struct BdrvAction {
+TransactionActionDrv *drv;
+void *opaque;
+} BdrvAction;
+
+void tran_prepend(GSList **list, TransactionActionDrv *drv, void *opaque)
+{
+BdrvAction *act;
+
+act = g_new(BdrvAction, 1);
+*act = (BdrvAction) {
+.drv = drv,
+.opaque = opaque
+};
+
+*list = g_slist_prepend(*list, act);
+}
+
+void tran_abort(GSList *list)
+{
+GSList *p;
+
+for (p = list; p != NULL; p = p->next) {
+BdrvAction *act = p->data;
+
+if (act->drv->abort) {
+act->drv->abort(act->opaque);
+}
+
+if (act->drv->clean) {
+act->drv->clean(act->opaque);
+}
+}
+
+g_slist_free_full(list, g_free);
+}
+
+void tran_commit(GSList *list)
+{
+GSList *p;
+
+for (p = list; p != NULL; p = p->next) {
+BdrvAction *act = p->data;
+
+if (act->drv->commit) {
+act->drv->commit(act->opaque);
+}
+
+if (act->drv->clean) {
+act->drv->clean(act->opaque);
+}
+}
+
+g_slist_free_full(list, g_free);
+}
diff --git a/util/meson.build b/util/meson.build
index f359af0d46..8c7c28bd40 100644
--- a/util/meson.build
+++ b/util/meson.build
@@ -41,6 +41,7 @@ util_ss.add(files('qsp.c'))
 util_ss.add(files('range.c'))
 util_ss.add(files('stats64.c'))

[PATCH v2 13/36] block: rewrite bdrv_child_try_set_perm() using bdrv_refresh_perms()

[Vladimir Sementsov-Ogievskiy]

We are going to drop recursive bdrv_child_* functions, so stop use them
in bdrv_child_try_set_perm() as a first step.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/block.c b/block.c
index 7267b4a3e9..82786ebe1f 100644
--- a/block.c
+++ b/block.c
@@ -2394,11 +2394,16 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t 
perm, uint64_t shared,
 Error **errp)
 {
 Error *local_err = NULL;
+GSList *tran = NULL;
 int ret;
 
-ret = bdrv_child_check_perm(c, NULL, perm, shared, NULL, _err);
+bdrv_child_set_perm_safe(c, perm, shared, );
+
+ret = bdrv_refresh_perms(c->bs, _err);
+
+tran_finalize(tran, ret);
+
 if (ret < 0) {
-bdrv_child_abort_perm_update(c);
 if ((perm & ~c->perm) || (c->shared_perm & ~shared)) {
 /* tighten permissions */
 error_propagate(errp, local_err);
@@ -2412,12 +2417,9 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, 
uint64_t shared,
 error_free(local_err);
 ret = 0;
 }
-return ret;
 }
 
-bdrv_child_set_perm(c);
-
-return 0;
+return ret;
 }
 
 int bdrv_child_refresh_perms(BlockDriverState *bs, BdrvChild *c, Error **errp)
-- 
2.21.3

[PATCH v2 15/36] block: use topological sort for permission update

[Vladimir Sementsov-Ogievskiy]

Rewrite bdrv_check_perm(), bdrv_abort_perm_update() and bdrv_set_perm()
to update nodes in topological sort order instead of simple DFS. With
topologically sorted nodes, we update a node only when all its parents
already updated. With DFS it's not so.

Consider the following example:

A -+
|  |
|  v
|  B
|  |
v  |
C<-+

A is parent for B and C, B is parent for C.

Obviously, to update permissions, we should go in order A B C, so, when
we update C, all parent permissions already updated. But with current
approach (simple recursion) we can update in sequence A C B C (C is
updated twice). On first update of C, we consider old B permissions, so
doing wrong thing. If it succeed, all is OK, on second C update we will
finish with correct graph. But if the wrong thing failed, we break the
whole process for no reason (it's possible that updated B permission
will be less strict, but we will never check it).

Also new approach gives a way to simultaneously and correctly update
several nodes, we just need to run bdrv_topological_dfs() several times
to add all nodes and their subtrees into one topologically sorted list
(next patch will update bdrv_replace_node() in this manner).

Test test_parallel_perm_update() is now passing, so move it out of
debugging "if".

We also need to support ignore_children in
bdrv_check_parents_compliance().

For test 283 order of parents compliance check is changed.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 103 +---
 tests/test-bdrv-graph-mod.c |   4 +-
 tests/qemu-iotests/283.out  |   2 +-
 3 files changed, 86 insertions(+), 23 deletions(-)

diff --git a/block.c b/block.c
index 92bfcbedc9..81ccf51605 100644
--- a/block.c
+++ b/block.c
@@ -1994,7 +1994,9 @@ static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, 
Error **errp)
 return false;
 }
 
-static bool bdrv_check_parents_compliance(BlockDriverState *bs, Error **errp)
+static bool bdrv_check_parents_compliance(BlockDriverState *bs,
+  GSList *ignore_children,
+  Error **errp)
 {
 BdrvChild *a, *b;
 
@@ -2005,7 +2007,9 @@ static bool 
bdrv_check_parents_compliance(BlockDriverState *bs, Error **errp)
  */
 QLIST_FOREACH(a, >parents, next_parent) {
 QLIST_FOREACH(b, >parents, next_parent) {
-if (a == b) {
+if (a == b || g_slist_find(ignore_children, a) ||
+g_slist_find(ignore_children, b))
+{
 continue;
 }
 
@@ -2034,6 +2038,29 @@ static void bdrv_child_perm(BlockDriverState *bs, 
BlockDriverState *child_bs,
 }
 }
 
+static GSList *bdrv_topological_dfs(GSList *list, GHashTable *found,
+BlockDriverState *bs)
+{
+BdrvChild *child;
+g_autoptr(GHashTable) local_found = NULL;
+
+if (!found) {
+assert(!list);
+found = local_found = g_hash_table_new(NULL, NULL);
+}
+
+if (g_hash_table_contains(found, bs)) {
+return list;
+}
+g_hash_table_add(found, bs);
+
+QLIST_FOREACH(child, >children, next) {
+list = bdrv_topological_dfs(list, found, child->bs);
+}
+
+return g_slist_prepend(list, bs);
+}
+
 static void bdrv_child_set_perm_commit(void *opaque)
 {
 BdrvChild *c = opaque;
@@ -2098,10 +2125,10 @@ static void bdrv_child_set_perm_safe(BdrvChild *c, 
uint64_t perm,
  * A call to this function must always be followed by a call to bdrv_set_perm()
  * or bdrv_abort_perm_update().
  */
-static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
-   uint64_t cumulative_perms,
-   uint64_t cumulative_shared_perms,
-   GSList *ignore_children, Error **errp)
+static int bdrv_node_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
+uint64_t cumulative_perms,
+uint64_t cumulative_shared_perms,
+GSList *ignore_children, Error **errp)
 {
 BlockDriver *drv = bs->drv;
 BdrvChild *c;
@@ -2166,21 +2193,43 @@ static int bdrv_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 /* Check all children */
 QLIST_FOREACH(c, >children, next) {
 uint64_t cur_perm, cur_shared;
-GSList *cur_ignore_children;
 
 bdrv_child_perm(bs, c->bs, c, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
+bdrv_child_set_perm_safe(c, cur_perm, cur_shared, NULL);
+}
+
+return 0;
+}
+
+static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
+   uint64_t cumulative_perms,
+   uint64_t cumulative_shared_perms,
+   GSList *ignore_children, Error **errp)
+{
+int ret;
+BlockDriverState *root = bs;
+

[PATCH v2 32/36] block: inline bdrv_check_perm_common()

[Vladimir Sementsov-Ogievskiy]

bdrv_check_perm_common() has only one caller, so no more sense in
"common".

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 32 +++-
 1 file changed, 3 insertions(+), 29 deletions(-)

diff --git a/block.c b/block.c
index 3ea04bbd8f..6c87ad0287 100644
--- a/block.c
+++ b/block.c
@@ -2308,33 +2308,13 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 return 0;
 }
 
-/*
- * If use_cumulative_perms is true, use cumulative_perms and
- * cumulative_shared_perms for first element of the list. Otherwise just 
refresh
- * all permissions.
- */
-static int bdrv_check_perm_common(GSList *list, BlockReopenQueue *q,
-  bool use_cumulative_perms,
-  uint64_t cumulative_perms,
-  uint64_t cumulative_shared_perms,
-  GSList **tran, Error **errp)
+static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q,
+   GSList **tran, Error **errp)
 {
 int ret;
+uint64_t cumulative_perms, cumulative_shared_perms;
 BlockDriverState *bs;
 
-if (use_cumulative_perms) {
-bs = list->data;
-
-ret = bdrv_node_check_perm(bs, q, cumulative_perms,
-   cumulative_shared_perms,
-   tran, errp);
-if (ret < 0) {
-return ret;
-}
-
-list = list->next;
-}
-
 for ( ; list; list = list->next) {
 bs = list->data;
 
@@ -2356,12 +2336,6 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 return 0;
 }
 
-static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q,
-   GSList **tran, Error **errp)
-{
-return bdrv_check_perm_common(list, q, false, 0, 0, tran, errp);
-}
-
 static void bdrv_node_set_perm(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
-- 
2.21.3

[PATCH v2 16/36] block: add bdrv_drv_set_perm transaction action

[Vladimir Sementsov-Ogievskiy]

Refactor calling driver callbacks to a separate transaction action to
be used later.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 70 -
 1 file changed, 54 insertions(+), 16 deletions(-)

diff --git a/block.c b/block.c
index 81ccf51605..4a43a33401 100644
--- a/block.c
+++ b/block.c
@@ -2116,6 +2116,54 @@ static void bdrv_child_set_perm_safe(BdrvChild *c, 
uint64_t perm,
 }
 }
 
+static void bdrv_drv_set_perm_commit(void *opaque)
+{
+BlockDriverState *bs = opaque;
+uint64_t cumulative_perms, cumulative_shared_perms;
+
+if (bs->drv->bdrv_set_perm) {
+bdrv_get_cumulative_perm(bs, _perms,
+ _shared_perms);
+bs->drv->bdrv_set_perm(bs, cumulative_perms, cumulative_shared_perms);
+}
+}
+
+static void bdrv_drv_set_perm_abort(void *opaque)
+{
+BlockDriverState *bs = opaque;
+
+if (bs->drv->bdrv_abort_perm_update) {
+bs->drv->bdrv_abort_perm_update(bs);
+}
+}
+
+TransactionActionDrv bdrv_drv_set_perm_drv = {
+.abort = bdrv_drv_set_perm_abort,
+.commit = bdrv_drv_set_perm_commit,
+};
+
+static int bdrv_drv_set_perm(BlockDriverState *bs, uint64_t perm,
+ uint64_t shared_perm, GSList **tran,
+ Error **errp)
+{
+if (!bs->drv) {
+return 0;
+}
+
+if (bs->drv->bdrv_check_perm) {
+int ret = bs->drv->bdrv_check_perm(bs, perm, shared_perm, errp);
+if (ret < 0) {
+return ret;
+}
+}
+
+if (tran) {
+tran_prepend(tran, _drv_set_perm_drv, bs);
+}
+
+return 0;
+}
+
 /*
  * Check whether permissions on this node can be changed in a way that
  * @cumulative_perms and @cumulative_shared_perms are the new cumulative
@@ -2176,12 +2224,10 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 return 0;
 }
 
-if (drv->bdrv_check_perm) {
-ret = drv->bdrv_check_perm(bs, cumulative_perms,
-   cumulative_shared_perms, errp);
-if (ret < 0) {
-return ret;
-}
+ret = bdrv_drv_set_perm(bs, cumulative_perms, cumulative_shared_perms, 
NULL,
+errp);
+if (ret < 0) {
+return ret;
 }
 
 /* Drivers that never have children can omit .bdrv_child_perm() */
@@ -2249,9 +2295,7 @@ static void bdrv_node_abort_perm_update(BlockDriverState 
*bs)
 return;
 }
 
-if (drv->bdrv_abort_perm_update) {
-drv->bdrv_abort_perm_update(bs);
-}
+bdrv_drv_set_perm_abort(bs);
 
 QLIST_FOREACH(c, >children, next) {
 bdrv_child_set_perm_abort(c);
@@ -2269,7 +2313,6 @@ static void bdrv_abort_perm_update(BlockDriverState *bs)
 
 static void bdrv_node_set_perm(BlockDriverState *bs)
 {
-uint64_t cumulative_perms, cumulative_shared_perms;
 BlockDriver *drv = bs->drv;
 BdrvChild *c;
 
@@ -2277,12 +2320,7 @@ static void bdrv_node_set_perm(BlockDriverState *bs)
 return;
 }
 
-bdrv_get_cumulative_perm(bs, _perms, _shared_perms);
-
-/* Update this node */
-if (drv->bdrv_set_perm) {
-drv->bdrv_set_perm(bs, cumulative_perms, cumulative_shared_perms);
-}
+bdrv_drv_set_perm_commit(bs);
 
 /* Drivers that never have children can omit .bdrv_child_perm() */
 if (!drv->bdrv_child_perm) {
-- 
2.21.3

[PATCH v2 34/36] block: refactor bdrv_child_set_perm_safe() transaction action

[Vladimir Sementsov-Ogievskiy]

Old interfaces dropped, nobody directly calls
bdrv_child_set_perm_abort() and bdrv_child_set_perm_commit(), so we can
use personal state structure for the action and stop exploiting
BdrvChild structure. Also, drop "_safe" suffix which is redundant now.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block_int.h |  5 
 block.c   | 63 ++-
 2 files changed, 22 insertions(+), 46 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 24a04ac2dc..1e509db867 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -796,11 +796,6 @@ struct BdrvChild {
  */
 uint64_t shared_perm;
 
-/* backup of permissions during permission update procedure */
-bool has_backup_perm;
-uint64_t backup_perm;
-uint64_t backup_shared_perm;
-
 /*
  * This link is frozen: the child can neither be replaced nor
  * detached from the parent.
diff --git a/block.c b/block.c
index 3093d20db8..1fde22e4f4 100644
--- a/block.c
+++ b/block.c
@@ -2070,59 +2070,40 @@ static GSList *bdrv_topological_dfs(GSList *list, 
GHashTable *found,
 return g_slist_prepend(list, bs);
 }
 
-static void bdrv_child_set_perm_commit(void *opaque)
-{
-BdrvChild *c = opaque;
-
-c->has_backup_perm = false;
-}
+typedef struct BdrvChildSetPermState {
+BdrvChild *child;
+uint64_t old_perm;
+uint64_t old_shared_perm;
+} BdrvChildSetPermState;
 
 static void bdrv_child_set_perm_abort(void *opaque)
 {
-BdrvChild *c = opaque;
-/*
- * We may have child->has_backup_perm unset at this point, as in case of
- * _check_ stage of permission update failure we may _check_ not the whole
- * subtree.  Still, _abort_ is called on the whole subtree anyway.
- */
-if (c->has_backup_perm) {
-c->perm = c->backup_perm;
-c->shared_perm = c->backup_shared_perm;
-c->has_backup_perm = false;
-}
+BdrvChildSetPermState *s = opaque;
+
+s->child->perm = s->old_perm;
+s->child->shared_perm = s->old_shared_perm;
 }
 
 static TransactionActionDrv bdrv_child_set_pem_drv = {
 .abort = bdrv_child_set_perm_abort,
-.commit = bdrv_child_set_perm_commit,
+.clean = g_free,
 };
 
-/*
- * With tran=NULL needs to be followed by direct call to either
- * bdrv_child_set_perm_commit() or bdrv_child_set_perm_abort().
- *
- * With non-NUll tran needs to be followed by tran_abort() or tran_commit()
- * instead.
- */
-static void bdrv_child_set_perm_safe(BdrvChild *c, uint64_t perm,
- uint64_t shared, GSList **tran)
+static void bdrv_child_set_perm(BdrvChild *c, uint64_t perm,
+uint64_t shared, GSList **tran)
 {
-if (!c->has_backup_perm) {
-c->has_backup_perm = true;
-c->backup_perm = c->perm;
-c->backup_shared_perm = c->shared_perm;
-}
-/*
- * Note: it's OK if c->has_backup_perm was already set, as we can find the
- * same c twice during check_perm procedure
- */
+BdrvChildSetPermState *s = g_new(BdrvChildSetPermState, 1);
+
+*s = (BdrvChildSetPermState) {
+.child = c,
+.old_perm = c->perm,
+.old_shared_perm = c->shared_perm,
+};
 
 c->perm = perm;
 c->shared_perm = shared;
 
-if (tran) {
-tran_prepend(tran, _child_set_pem_drv, c);
-}
+tran_prepend(tran, _child_set_pem_drv, s);
 }
 
 static void bdrv_drv_set_perm_commit(void *opaque)
@@ -2302,7 +2283,7 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 bdrv_child_perm(bs, c->bs, c, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
-bdrv_child_set_perm_safe(c, cur_perm, cur_shared, tran);
+bdrv_child_set_perm(c, cur_perm, cur_shared, tran);
 }
 
 return 0;
@@ -2401,7 +2382,7 @@ int bdrv_child_try_set_perm(BdrvChild *c, uint64_t perm, 
uint64_t shared,
 GSList *tran = NULL;
 int ret;
 
-bdrv_child_set_perm_safe(c, perm, shared, );
+bdrv_child_set_perm(c, perm, shared, );
 
 ret = bdrv_refresh_perms(c->bs, _err);
 
-- 
2.21.3

[PATCH v2 02/36] tests/test-bdrv-graph-mod: add test_parallel_perm_update

[Vladimir Sementsov-Ogievskiy]

Add test to show that simple DFS recursion order is not correct for
permission update. Correct order is topological-sort order, which will
be introduced later.

Consider the block driver which has two filter children: one active
with exclusive write access and one inactive with no specific
permissions.

And, these two children has a common base child, like this:

┌─┐ ┌──┐
│ fl2 │ ◀── │ top  │
└─┘ └──┘
  │   │
  │   │ w
  │   ▼
  │ ┌──┐
  │ │ fl1  │
  │ └──┘
  │   │
  │   │ w
  │   ▼
  │ ┌──┐
  └───▶ │ base │
└──┘

So, exclusive write is propagated.

Assume, we want to make fl2 active instead of fl1.
So, we set some option for top driver and do permission update.

If permission update (remember, it's DFS) goes first through
top->fl1->base branch it will succeed: it firstly drop exclusive write
permissions and than apply them for another BdrvChildren.
But if permission update goes first through top->fl2->base branch it
will fail, as when we try to update fl2->base child, old not yet
updated fl1->base child will be in conflict.

Now test fails, so it runs only with -d flag. To run do

  ./test-bdrv-graph-mod -d -p /bdrv-graph-mod/parallel-perm-update

from /tests.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/test-bdrv-graph-mod.c | 64 +
 1 file changed, 64 insertions(+)

diff --git a/tests/test-bdrv-graph-mod.c b/tests/test-bdrv-graph-mod.c
index 3b9e6f242f..27e3361a60 100644
--- a/tests/test-bdrv-graph-mod.c
+++ b/tests/test-bdrv-graph-mod.c
@@ -232,6 +232,68 @@ static void test_parallel_exclusive_write(void)
 bdrv_unref(top);
 }
 
+static void write_to_file_perms(BlockDriverState *bs, BdrvChild *c,
+ BdrvChildRole role,
+ BlockReopenQueue *reopen_queue,
+ uint64_t perm, uint64_t shared,
+ uint64_t *nperm, uint64_t *nshared)
+{
+if (bs->file && c == bs->file) {
+*nperm = BLK_PERM_WRITE;
+*nshared = BLK_PERM_ALL & ~BLK_PERM_WRITE;
+} else {
+*nperm = 0;
+*nshared = BLK_PERM_ALL;
+}
+}
+
+static BlockDriver bdrv_write_to_file = {
+.format_name = "tricky-perm",
+.bdrv_child_perm = write_to_file_perms,
+};
+
+static void test_parallel_perm_update(void)
+{
+BlockDriverState *top = no_perm_node("top");
+BlockDriverState *tricky =
+bdrv_new_open_driver(_write_to_file, "tricky", BDRV_O_RDWR,
+ _abort);
+BlockDriverState *base = no_perm_node("base");
+BlockDriverState *fl1 = pass_through_node("fl1");
+BlockDriverState *fl2 = pass_through_node("fl2");
+BdrvChild *c_fl1, *c_fl2;
+
+bdrv_attach_child(top, tricky, "file", _of_bds, BDRV_CHILD_DATA,
+  _abort);
+c_fl1 = bdrv_attach_child(tricky, fl1, "first", _of_bds,
+  BDRV_CHILD_FILTERED, _abort);
+c_fl2 = bdrv_attach_child(tricky, fl2, "second", _of_bds,
+  BDRV_CHILD_FILTERED, _abort);
+bdrv_attach_child(fl1, base, "backing", _of_bds, BDRV_CHILD_FILTERED,
+  _abort);
+bdrv_attach_child(fl2, base, "backing", _of_bds, BDRV_CHILD_FILTERED,
+  _abort);
+bdrv_ref(base);
+
+/* Select fl1 as first child to be active */
+tricky->file = c_fl1;
+bdrv_child_refresh_perms(top, top->children.lh_first, _abort);
+
+assert(c_fl1->perm & BLK_PERM_WRITE);
+
+/* Now, try to switch active child and update permissions */
+tricky->file = c_fl2;
+bdrv_child_refresh_perms(top, top->children.lh_first, _abort);
+
+assert(c_fl2->perm & BLK_PERM_WRITE);
+
+/* Switch once more, to not care about real child order in the list */
+tricky->file = c_fl1;
+bdrv_child_refresh_perms(top, top->children.lh_first, _abort);
+
+assert(c_fl1->perm & BLK_PERM_WRITE);
+}
+
 int main(int argc, char *argv[])
 {
 int i;
@@ -256,6 +318,8 @@ int main(int argc, char *argv[])
 if (debug) {
 g_test_add_func("/bdrv-graph-mod/parallel-exclusive-write",
 test_parallel_exclusive_write);
+g_test_add_func("/bdrv-graph-mod/parallel-perm-update",
+test_parallel_perm_update);
 }
 
 return g_test_run();
-- 
2.21.3

[PATCH v2 22/36] block: split out bdrv_replace_node_noperm()

[Vladimir Sementsov-Ogievskiy]

Split part of bdrv_replace_node_common() to be used separately.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 47 ++-
 1 file changed, 30 insertions(+), 17 deletions(-)

diff --git a/block.c b/block.c
index 162a247579..02da1a90bc 100644
--- a/block.c
+++ b/block.c
@@ -4897,6 +4897,33 @@ static bool should_update_child(BdrvChild *c, 
BlockDriverState *to)
 return ret;
 }
 
+static int bdrv_replace_node_noperm(BlockDriverState *from,
+BlockDriverState *to,
+bool auto_skip, GSList **tran, Error 
**errp)
+{
+BdrvChild *c, *next;
+
+QLIST_FOREACH_SAFE(c, >parents, next_parent, next) {
+assert(c->bs == from);
+if (!should_update_child(c, to)) {
+if (auto_skip) {
+continue;
+}
+error_setg(errp, "Should not change '%s' link to '%s'",
+   c->name, from->node_name);
+return -EPERM;
+}
+if (c->frozen) {
+error_setg(errp, "Cannot change '%s' link to '%s'",
+   c->name, from->node_name);
+return -EPERM;
+}
+bdrv_replace_child_safe(c, to, tran);
+}
+
+return 0;
+}
+
 /*
  * With auto_skip=true bdrv_replace_node_common skips updating from parents
  * if it creates a parent-child relation loop or if parent is block-job.
@@ -4909,7 +4936,6 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
 bool auto_skip, Error **errp)
 {
 int ret = -EPERM;
-BdrvChild *c, *next;
 GSList *tran = NULL;
 g_autoptr(GHashTable) found = NULL;
 g_autoptr(GSList) refresh_list = NULL;
@@ -4928,22 +4954,9 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
  * permissions based on new graph. If we fail, we'll roll-back the
  * replacement.
  */
-QLIST_FOREACH_SAFE(c, >parents, next_parent, next) {
-assert(c->bs == from);
-if (!should_update_child(c, to)) {
-if (auto_skip) {
-continue;
-}
-error_setg(errp, "Should not change '%s' link to '%s'",
-   c->name, from->node_name);
-goto out;
-}
-if (c->frozen) {
-error_setg(errp, "Cannot change '%s' link to '%s'",
-   c->name, from->node_name);
-goto out;
-}
-bdrv_replace_child_safe(c, to, );
+ret = bdrv_replace_node_noperm(from, to, auto_skip, , errp);
+if (ret < 0) {
+goto out;
 }
 
 found = g_hash_table_new(NULL, NULL);
-- 
2.21.3

[PATCH v2 28/36] block: add bdrv_set_backing_noperm() transaction action

[Vladimir Sementsov-Ogievskiy]

Split out no-perm part of bdrv_set_backing_hd() as a separate
transaction action. Note the in case of existing BdrvChild we reuse it,
not recreate, just to do less actions.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 111 +---
 1 file changed, 89 insertions(+), 22 deletions(-)

diff --git a/block.c b/block.c
index 54fb6d24bd..617cba9547 100644
--- a/block.c
+++ b/block.c
@@ -101,6 +101,7 @@ static int bdrv_attach_child_common(BlockDriverState 
*child_bs,
 uint64_t perm, uint64_t shared_perm,
 void *opaque, BdrvChild **child,
 GSList **tran, Error **errp);
+static void bdrv_remove_backing(BlockDriverState *bs, GSList **tran);
 
 static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue
*queue, Error **errp);
@@ -3194,45 +3195,111 @@ static BdrvChildRole 
bdrv_backing_role(BlockDriverState *bs)
 }
 }
 
+typedef struct BdrvSetBackingNoPermState {
+BlockDriverState *bs;
+BlockDriverState *backing_bs;
+BlockDriverState *old_inherits_from;
+GSList *attach_tran;
+} BdrvSetBackingNoPermState;
+
+static void bdrv_set_backing_noperm_abort(void *opaque)
+{
+BdrvSetBackingNoPermState *s = opaque;
+
+if (s->backing_bs) {
+s->backing_bs->inherits_from = s->old_inherits_from;
+}
+
+tran_abort(s->attach_tran);
+
+bdrv_refresh_limits(s->bs, NULL);
+if (s->old_inherits_from) {
+bdrv_refresh_limits(s->old_inherits_from, NULL);
+}
+}
+
+static void bdrv_set_backing_noperm_commit(void *opaque)
+{
+BdrvSetBackingNoPermState *s = opaque;
+
+tran_commit(s->attach_tran);
+}
+
+static TransactionActionDrv bdrv_set_backing_noperm_drv = {
+.abort = bdrv_set_backing_noperm_abort,
+.commit = bdrv_set_backing_noperm_commit,
+.clean = g_free,
+};
+
 /*
  * Sets the bs->backing link of a BDS. A new reference is created; callers
  * which don't need their own reference any more must call bdrv_unref().
  */
-void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
- Error **errp)
+static int bdrv_set_backing_noperm(BlockDriverState *bs,
+   BlockDriverState *backing_bs,
+   GSList **tran, Error **errp)
 {
-bool update_inherits_from = bdrv_chain_contains(bs, backing_hd) &&
-bdrv_inherits_from_recursive(backing_hd, bs);
+int ret = 0;
+bool update_inherits_from = bdrv_chain_contains(bs, backing_bs) &&
+bdrv_inherits_from_recursive(backing_bs, bs);
+GSList *attach_tran = NULL;
+BdrvSetBackingNoPermState *s;
 
 if (bdrv_is_backing_chain_frozen(bs, child_bs(bs->backing), errp)) {
-return;
+return -EPERM;
 }
 
-if (backing_hd) {
-bdrv_ref(backing_hd);
+if (bs->backing && backing_bs) {
+bdrv_replace_child_safe(bs->backing, backing_bs, tran);
+} else if (bs->backing && !backing_bs) {
+bdrv_remove_backing(bs, tran);
+} else if (backing_bs) {
+assert(!bs->backing);
+ret = bdrv_attach_child_noperm(bs, backing_bs, "backing",
+   _of_bds, bdrv_backing_role(bs),
+   >backing, _tran, errp);
+if (ret < 0) {
+tran_abort(attach_tran);
+return ret;
+}
 }
 
-if (bs->backing) {
-/* Cannot be frozen, we checked that above */
-bdrv_unref_child(bs, bs->backing);
-bs->backing = NULL;
-}
+s = g_new(BdrvSetBackingNoPermState, 1);
+*s = (BdrvSetBackingNoPermState) {
+.bs = bs,
+.backing_bs = backing_bs,
+.old_inherits_from = backing_bs ? backing_bs->inherits_from : NULL,
+};
+tran_prepend(tran, _set_backing_noperm_drv, s);
 
-if (!backing_hd) {
-goto out;
+/*
+ * If backing_bs was already part of bs's backing chain, and
+ * inherits_from pointed recursively to bs then let's update it to
+ * point directly to bs (else it will become NULL).
+ */
+if (backing_bs && update_inherits_from) {
+backing_bs->inherits_from = bs;
 }
 
-bs->backing = bdrv_attach_child(bs, backing_hd, "backing", _of_bds,
-bdrv_backing_role(bs), errp);
-/* If backing_hd was already part of bs's backing chain, and
- * inherits_from pointed recursively to bs then let's update it to
- * point directly to bs (else it will become NULL). */
-if (bs->backing && update_inherits_from) {
-backing_hd->inherits_from = bs;
+bdrv_refresh_limits(bs, NULL);
+
+return 0;
+}
+
+void bdrv_set_backing_hd(BlockDriverState *bs, BlockDriverState *backing_hd,
+ Error **errp)
+{
+int ret;
+GSList *tran = NULL;
+
+ret = bdrv_set_backing_noperm(bs, backing_hd, , 

[PATCH v2 26/36] block/backup-top: drop .active

[Vladimir Sementsov-Ogievskiy]

We don't need this workaround anymore: bdrv_append is already smart
enough and we can use new bdrv_drop_filter().

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/backup-top.c | 38 +-
 tests/qemu-iotests/283.out |  2 +-
 2 files changed, 2 insertions(+), 38 deletions(-)

diff --git a/block/backup-top.c b/block/backup-top.c
index 650ed6195c..84eb73aeb7 100644
--- a/block/backup-top.c
+++ b/block/backup-top.c
@@ -37,7 +37,6 @@
 typedef struct BDRVBackupTopState {
 BlockCopyState *bcs;
 BdrvChild *target;
-bool active;
 int64_t cluster_size;
 } BDRVBackupTopState;
 
@@ -127,21 +126,6 @@ static void backup_top_child_perm(BlockDriverState *bs, 
BdrvChild *c,
   uint64_t perm, uint64_t shared,
   uint64_t *nperm, uint64_t *nshared)
 {
-BDRVBackupTopState *s = bs->opaque;
-
-if (!s->active) {
-/*
- * The filter node may be in process of bdrv_append(), which firstly do
- * bdrv_set_backing_hd() and then bdrv_replace_node(). This means that
- * we can't unshare BLK_PERM_WRITE during bdrv_append() operation. So,
- * let's require nothing during bdrv_append() and refresh permissions
- * after it (see bdrv_backup_top_append()).
- */
-*nperm = 0;
-*nshared = BLK_PERM_ALL;
-return;
-}
-
 if (!(role & BDRV_CHILD_FILTERED)) {
 /*
  * Target child
@@ -229,18 +213,6 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState 
*source,
 }
 appended = true;
 
-/*
- * bdrv_append() finished successfully, now we can require permissions
- * we want.
- */
-state->active = true;
-bdrv_child_refresh_perms(top, top->backing, _err);
-if (local_err) {
-error_prepend(_err,
-  "Cannot set permissions for backup-top filter: ");
-goto fail;
-}
-
 state->cluster_size = cluster_size;
 state->bcs = block_copy_state_new(top->backing, state->target,
   cluster_size, write_flags, _err);
@@ -256,7 +228,6 @@ BlockDriverState *bdrv_backup_top_append(BlockDriverState 
*source,
 
 fail:
 if (appended) {
-state->active = false;
 bdrv_backup_top_drop(top);
 } else {
 bdrv_unref(top);
@@ -272,16 +243,9 @@ void bdrv_backup_top_drop(BlockDriverState *bs)
 {
 BDRVBackupTopState *s = bs->opaque;
 
-bdrv_drained_begin(bs);
+bdrv_drop_filter(bs, _abort);
 
 block_copy_state_free(s->bcs);
 
-s->active = false;
-bdrv_child_refresh_perms(bs, bs->backing, _abort);
-bdrv_replace_node(bs, bs->backing->bs, _abort);
-bdrv_set_backing_hd(bs, NULL, _abort);
-
-bdrv_drained_end(bs);
-
 bdrv_unref(bs);
 }
diff --git a/tests/qemu-iotests/283.out b/tests/qemu-iotests/283.out
index fbb7d0f619..a34e4e3f92 100644
--- a/tests/qemu-iotests/283.out
+++ b/tests/qemu-iotests/283.out
@@ -5,4 +5,4 @@
 {"execute": "blockdev-add", "arguments": {"driver": "blkdebug", "image": 
"base", "node-name": "other", "take-child-perms": ["write"]}}
 {"return": {}}
 {"execute": "blockdev-backup", "arguments": {"device": "source", "sync": 
"full", "target": "target"}}
-{"error": {"class": "GenericError", "desc": "Cannot set permissions for 
backup-top filter: Conflicts with use by source as 'image', which does not 
allow 'write' on base"}}
+{"error": {"class": "GenericError", "desc": "Cannot append backup-top filter: 
Conflicts with use by source as 'image', which does not allow 'write' on base"}}
-- 
2.21.3

[PATCH v2 29/36] blockdev: qmp_x_blockdev_reopen: acquire all contexts

[Vladimir Sementsov-Ogievskiy]

During reopen we may add backing bs from other aio context, which may
lead to changing original context of top bs.

We are going to move graph modification to prepare stage. So, it will
be possible that bdrv_flush() in bdrv_reopen_prepare called on bs in
non-original aio context, which we didn't aquire which leads to crash.

More correct would be to acquire all aio context we are going to work
with. And the simplest ways is to just acquire all of them. It may be
optimized later if needed.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 blockdev.c | 23 +++
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 2af35d0958..098a05709d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3531,7 +3531,6 @@ fail:
 void qmp_x_blockdev_reopen(BlockdevOptions *options, Error **errp)
 {
 BlockDriverState *bs;
-AioContext *ctx;
 QObject *obj;
 Visitor *v = qobject_output_visitor_new();
 BlockReopenQueue *queue;
@@ -3557,13 +3556,29 @@ void qmp_x_blockdev_reopen(BlockdevOptions *options, 
Error **errp)
 qdict_flatten(qdict);
 
 /* Perform the reopen operation */
-ctx = bdrv_get_aio_context(bs);
-aio_context_acquire(ctx);
+BdrvNextIterator it;
+GSList *aio_ctxs = NULL, *ctx;
+BlockDriverState *it_bs;
+
+for (it_bs = bdrv_first(); it_bs; it_bs = bdrv_next()) {
+AioContext *aio_context = bdrv_get_aio_context(it_bs);
+
+if (!g_slist_find(aio_ctxs, aio_context)) {
+aio_ctxs = g_slist_prepend(aio_ctxs, aio_context);
+aio_context_acquire(aio_context);
+}
+}
+
 bdrv_subtree_drained_begin(bs);
 queue = bdrv_reopen_queue(NULL, bs, qdict, false);
 bdrv_reopen_multiple(queue, errp);
 bdrv_subtree_drained_end(bs);
-aio_context_release(ctx);
+
+for (ctx = aio_ctxs; ctx != NULL; ctx = ctx->next) {
+AioContext *aio_context = ctx->data;
+aio_context_release(aio_context);
+}
+g_slist_free(aio_ctxs);
 
 fail:
 visit_free(v);
-- 
2.21.3

[PATCH v2 23/36] block: adapt bdrv_append() for inserting filters

[Vladimir Sementsov-Ogievskiy]

bdrv_append is not very good for inserting filters: it does extra
permission update as part of bdrv_set_backing_hd(). During this update
filter may conflict with other parents of top_bs.

Instead, let's first do all graph modifications and after it update
permissions.

Note: bdrv_append() is still only works for backing-child based
filters. It's something to improve later.

It simplifies the fact that bdrv_append() used to append new nodes,
without backing child. Let's add an assertion.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 28 +---
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/block.c b/block.c
index 02da1a90bc..7094922509 100644
--- a/block.c
+++ b/block.c
@@ -4998,22 +4998,28 @@ int bdrv_replace_node(BlockDriverState *from, 
BlockDriverState *to,
 int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
 Error **errp)
 {
-Error *local_err = NULL;
+int ret;
+GSList *tran = NULL;
 
-bdrv_set_backing_hd(bs_new, bs_top, _err);
-if (local_err) {
-error_propagate(errp, local_err);
-return -EPERM;
+assert(!bs_new->backing);
+
+ret = bdrv_attach_child_noperm(bs_new, bs_top, "backing",
+   _of_bds, bdrv_backing_role(bs_new),
+   _new->backing, , errp);
+if (ret < 0) {
+goto out;
 }
 
-bdrv_replace_node(bs_top, bs_new, _err);
-if (local_err) {
-error_propagate(errp, local_err);
-bdrv_set_backing_hd(bs_new, NULL, _abort);
-return -EPERM;
+ret = bdrv_replace_node_noperm(bs_top, bs_new, true, , errp);
+if (ret < 0) {
+goto out;
 }
 
-return 0;
+ret = bdrv_refresh_perms(bs_new, errp);
+out:
+tran_finalize(tran, ret);
+
+return ret;
 }
 
 static void bdrv_delete(BlockDriverState *bs)
-- 
2.21.3

[PATCH v2 33/36] block: inline bdrv_replace_child()

[Vladimir Sementsov-Ogievskiy]

bdrv_replace_child() has only one caller, the second argument is
unused. Inline it now. This triggers deletion of some more unused
interfaces.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 101 ++--
 1 file changed, 18 insertions(+), 83 deletions(-)

diff --git a/block.c b/block.c
index 6c87ad0287..3093d20db8 100644
--- a/block.c
+++ b/block.c
@@ -2336,42 +2336,6 @@ static int bdrv_list_refresh_perms(GSList *list, 
BlockReopenQueue *q,
 return 0;
 }
 
-static void bdrv_node_set_perm(BlockDriverState *bs)
-{
-BlockDriver *drv = bs->drv;
-BdrvChild *c;
-
-if (!drv) {
-return;
-}
-
-bdrv_drv_set_perm_commit(bs);
-
-/* Drivers that never have children can omit .bdrv_child_perm() */
-if (!drv->bdrv_child_perm) {
-assert(QLIST_EMPTY(>children));
-return;
-}
-
-/* Update all children */
-QLIST_FOREACH(c, >children, next) {
-bdrv_child_set_perm_commit(c);
-}
-}
-
-static void bdrv_list_set_perm(GSList *list)
-{
-for ( ; list; list = list->next) {
-bdrv_node_set_perm((BlockDriverState *)list->data);
-}
-}
-
-static void bdrv_set_perm(BlockDriverState *bs)
-{
-g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
-return bdrv_list_set_perm(list);
-}
-
 void bdrv_get_cumulative_perm(BlockDriverState *bs, uint64_t *perm,
   uint64_t *shared_perm)
 {
@@ -2711,52 +2675,6 @@ static void bdrv_replace_child_noperm(BdrvChild *child,
 }
 }
 
-/*
- * Updates @child to change its reference to point to @new_bs, including
- * checking and applying the necessary permission updates both to the old node
- * and to @new_bs.
- *
- * NULL is passed as @new_bs for removing the reference before freeing @child.
- *
- * If @new_bs is not NULL, bdrv_check_perm() must be called beforehand, as this
- * function uses bdrv_set_perm() to update the permissions according to the new
- * reference that @new_bs gets.
- *
- * Callers must ensure that child->frozen is false.
- */
-static void bdrv_replace_child(BdrvChild *child, BlockDriverState *new_bs)
-{
-BlockDriverState *old_bs = child->bs;
-
-/* Asserts that child->frozen == false */
-bdrv_replace_child_noperm(child, new_bs);
-
-/*
- * Start with the new node's permissions.  If @new_bs is a (direct
- * or indirect) child of @old_bs, we must complete the permission
- * update on @new_bs before we loosen the restrictions on @old_bs.
- * Otherwise, bdrv_check_perm() on @old_bs would re-initiate
- * updating the permissions of @new_bs, and thus not purely loosen
- * restrictions.
- */
-if (new_bs) {
-bdrv_set_perm(new_bs);
-}
-
-if (old_bs) {
-/*
- * Update permissions for old node. We're just taking a parent away, so
- * we're loosening restrictions. Errors of permission update are not
- * fatal in this case, ignore them.
- */
-bdrv_refresh_perms(old_bs, NULL);
-
-/* When the parent requiring a non-default AioContext is removed, the
- * node moves back to the main AioContext */
-bdrv_try_set_aio_context(old_bs, qemu_get_aio_context(), NULL);
-}
-}
-
 /*
  * This function steals the reference to child_bs from the caller.
  * That reference is later dropped by bdrv_root_unref_child().
@@ -2979,8 +2897,25 @@ static int bdrv_attach_child_noperm(BlockDriverState 
*parent_bs,
 
 static void bdrv_detach_child(BdrvChild *child)
 {
-bdrv_replace_child(child, NULL);
+BlockDriverState *old_bs = child->bs;
+
+bdrv_replace_child_noperm(child, NULL);
 bdrv_remove_empty_child(child);
+
+if (old_bs) {
+/*
+ * Update permissions for old node. We're just taking a parent away, so
+ * we're loosening restrictions. Errors of permission update are not
+ * fatal in this case, ignore them.
+ */
+bdrv_refresh_perms(old_bs, NULL);
+
+/*
+ * When the parent requiring a non-default AioContext is removed, the
+ * node moves back to the main AioContext
+ */
+bdrv_try_set_aio_context(old_bs, qemu_get_aio_context(), NULL);
+}
 }
 
 /* Callers must ensure that child->frozen is false. */
-- 
2.21.3

[PATCH v2 05/36] block: add bdrv_parent_try_set_aio_context

[Vladimir Sementsov-Ogievskiy]

We already have bdrv_parent_can_set_aio_context(). Add corresponding
bdrv_parent_set_aio_context_ignore() and
bdrv_parent_try_set_aio_context() and use them instead of open-coding.

Make bdrv_parent_try_set_aio_context() public, as it will be used in
further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |  2 ++
 block.c   | 51 +--
 2 files changed, 41 insertions(+), 12 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index ee3f5a6cca..550c5a7513 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -686,6 +686,8 @@ bool bdrv_child_can_set_aio_context(BdrvChild *c, 
AioContext *ctx,
 GSList **ignore, Error **errp);
 bool bdrv_can_set_aio_context(BlockDriverState *bs, AioContext *ctx,
   GSList **ignore, Error **errp);
+int bdrv_parent_try_set_aio_context(BdrvChild *c, AioContext *ctx,
+Error **errp);
 int bdrv_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz);
 int bdrv_probe_geometry(BlockDriverState *bs, HDGeometry *geo);
 
diff --git a/block.c b/block.c
index 916087ee1a..5d925c208d 100644
--- a/block.c
+++ b/block.c
@@ -81,6 +81,9 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
BdrvChildRole child_role,
Error **errp);
 
+static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, AioContext *ctx,
+   GSList **ignore);
+
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
 
@@ -2655,17 +2658,12 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
  * try moving the parent into the AioContext of child_bs instead. */
 if (bdrv_get_aio_context(child_bs) != ctx) {
 ret = bdrv_try_set_aio_context(child_bs, ctx, _err);
-if (ret < 0 && child_class->can_set_aio_ctx) {
-GSList *ignore = g_slist_prepend(NULL, child);
-ctx = bdrv_get_aio_context(child_bs);
-if (child_class->can_set_aio_ctx(child, ctx, , NULL)) {
-error_free(local_err);
+if (ret < 0) {
+if (bdrv_parent_try_set_aio_context(child, ctx, NULL) == 0) {
 ret = 0;
-g_slist_free(ignore);
-ignore = g_slist_prepend(NULL, child);
-child_class->set_aio_ctx(child, ctx, );
+error_free(local_err);
+local_err = NULL;
 }
-g_slist_free(ignore);
 }
 if (ret < 0) {
 error_propagate(errp, local_err);
@@ -6452,9 +6450,7 @@ void bdrv_set_aio_context_ignore(BlockDriverState *bs,
 if (g_slist_find(*ignore, child)) {
 continue;
 }
-assert(child->klass->set_aio_ctx);
-*ignore = g_slist_prepend(*ignore, child);
-child->klass->set_aio_ctx(child, new_context, ignore);
+bdrv_parent_set_aio_context_ignore(child, new_context, ignore);
 }
 
 bdrv_detach_aio_context(bs);
@@ -6511,6 +6507,37 @@ static bool bdrv_parent_can_set_aio_context(BdrvChild 
*c, AioContext *ctx,
 return true;
 }
 
+static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, AioContext *ctx,
+   GSList **ignore)
+{
+if (g_slist_find(*ignore, c)) {
+return;
+}
+*ignore = g_slist_prepend(*ignore, c);
+
+assert(c->klass->set_aio_ctx);
+c->klass->set_aio_ctx(c, ctx, ignore);
+}
+
+int bdrv_parent_try_set_aio_context(BdrvChild *c, AioContext *ctx,
+Error **errp)
+{
+GSList *ignore = NULL;
+
+if (!bdrv_parent_can_set_aio_context(c, ctx, , errp)) {
+g_slist_free(ignore);
+return -EPERM;
+}
+
+g_slist_free(ignore);
+ignore = NULL;
+
+bdrv_parent_set_aio_context_ignore(c, ctx, );
+g_slist_free(ignore);
+
+return 0;
+}
+
 bool bdrv_child_can_set_aio_context(BdrvChild *c, AioContext *ctx,
 GSList **ignore, Error **errp)
 {
-- 
2.21.3

[PATCH v2 12/36] block: refactor bdrv_child* permission functions

[Vladimir Sementsov-Ogievskiy]

Split out non-recursive parts, and refactor as block graph transaction
action.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 79 ++---
 1 file changed, 59 insertions(+), 20 deletions(-)

diff --git a/block.c b/block.c
index a756f3e8ad..7267b4a3e9 100644
--- a/block.c
+++ b/block.c
@@ -48,6 +48,7 @@
 #include "qemu/timer.h"
 #include "qemu/cutils.h"
 #include "qemu/id.h"
+#include "qemu/transactions.h"
 #include "block/coroutines.h"
 
 #ifdef CONFIG_BSD
@@ -2033,6 +2034,61 @@ static void bdrv_child_perm(BlockDriverState *bs, 
BlockDriverState *child_bs,
 }
 }
 
+static void bdrv_child_set_perm_commit(void *opaque)
+{
+BdrvChild *c = opaque;
+
+c->has_backup_perm = false;
+}
+
+static void bdrv_child_set_perm_abort(void *opaque)
+{
+BdrvChild *c = opaque;
+/*
+ * We may have child->has_backup_perm unset at this point, as in case of
+ * _check_ stage of permission update failure we may _check_ not the whole
+ * subtree.  Still, _abort_ is called on the whole subtree anyway.
+ */
+if (c->has_backup_perm) {
+c->perm = c->backup_perm;
+c->shared_perm = c->backup_shared_perm;
+c->has_backup_perm = false;
+}
+}
+
+static TransactionActionDrv bdrv_child_set_pem_drv = {
+.abort = bdrv_child_set_perm_abort,
+.commit = bdrv_child_set_perm_commit,
+};
+
+/*
+ * With tran=NULL needs to be followed by direct call to either
+ * bdrv_child_set_perm_commit() or bdrv_child_set_perm_abort().
+ *
+ * With non-NUll tran needs to be followed by tran_abort() or tran_commit()
+ * instead.
+ */
+static void bdrv_child_set_perm_safe(BdrvChild *c, uint64_t perm,
+ uint64_t shared, GSList **tran)
+{
+if (!c->has_backup_perm) {
+c->has_backup_perm = true;
+c->backup_perm = c->perm;
+c->backup_shared_perm = c->shared_perm;
+}
+/*
+ * Note: it's OK if c->has_backup_perm was already set, as we can find the
+ * same c twice during check_perm procedure
+ */
+
+c->perm = perm;
+c->shared_perm = shared;
+
+if (tran) {
+tran_prepend(tran, _child_set_pem_drv, c);
+}
+}
+
 /*
  * Check whether permissions on this node can be changed in a way that
  * @cumulative_perms and @cumulative_shared_perms are the new cumulative
@@ -2298,37 +2354,20 @@ static int bdrv_child_check_perm(BdrvChild *c, 
BlockReopenQueue *q,
 return ret;
 }
 
-if (!c->has_backup_perm) {
-c->has_backup_perm = true;
-c->backup_perm = c->perm;
-c->backup_shared_perm = c->shared_perm;
-}
-/*
- * Note: it's OK if c->has_backup_perm was already set, as we can find the
- * same child twice during check_perm procedure
- */
-
-c->perm = perm;
-c->shared_perm = shared;
+bdrv_child_set_perm_safe(c, perm, shared, NULL);
 
 return 0;
 }
 
 static void bdrv_child_set_perm(BdrvChild *c)
 {
-c->has_backup_perm = false;
-
+bdrv_child_set_perm_commit(c);
 bdrv_set_perm(c->bs);
 }
 
 static void bdrv_child_abort_perm_update(BdrvChild *c)
 {
-if (c->has_backup_perm) {
-c->perm = c->backup_perm;
-c->shared_perm = c->backup_shared_perm;
-c->has_backup_perm = false;
-}
-
+bdrv_child_set_perm_abort(c);
 bdrv_abort_perm_update(c->bs);
 }
 
-- 
2.21.3

[PATCH v2 30/36] block: bdrv_reopen_multiple: refresh permissions on updated graph

[Vladimir Sementsov-Ogievskiy]

Move bdrv_reopen_multiple to new paradigm of permission update:
first update graph relations, then do refresh the permissions.

We have to modify reopen process in file-posix driver: with new scheme
we don't have prepared permissions in raw_reopen_prepare(), so we
should reconfigure fd in raw_check_perm(). Still this seems more native
and simple anyway.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |   2 +-
 block.c   | 183 +++---
 block/file-posix.c|  84 +--
 3 files changed, 70 insertions(+), 199 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 0f21ef313f..82271d9ccd 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -195,7 +195,7 @@ typedef struct BDRVReopenState {
 BlockdevDetectZeroesOptions detect_zeroes;
 bool backing_missing;
 bool replace_backing_bs;  /* new_backing_bs is ignored if this is false */
-BlockDriverState *new_backing_bs; /* If NULL then detach the current bs */
+BlockDriverState *old_backing_bs; /* keep pointer for permissions update */
 uint64_t perm, shared_perm;
 QDict *options;
 QDict *explicit_options;
diff --git a/block.c b/block.c
index 617cba9547..474e624152 100644
--- a/block.c
+++ b/block.c
@@ -103,8 +103,9 @@ static int bdrv_attach_child_common(BlockDriverState 
*child_bs,
 GSList **tran, Error **errp);
 static void bdrv_remove_backing(BlockDriverState *bs, GSList **tran);
 
-static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue
-   *queue, Error **errp);
+static int bdrv_reopen_prepare(BDRVReopenState *reopen_state,
+   BlockReopenQueue *queue,
+   GSList **set_backings_tran, Error **errp);
 static void bdrv_reopen_commit(BDRVReopenState *reopen_state);
 static void bdrv_reopen_abort(BDRVReopenState *reopen_state);
 
@@ -2403,6 +2404,7 @@ static void bdrv_list_abort_perm_update(GSList *list)
 }
 }
 
+__attribute__((unused))
 static void bdrv_abort_perm_update(BlockDriverState *bs)
 {
 g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
@@ -2498,6 +2500,7 @@ char *bdrv_perm_names(uint64_t perm)
  *
  * Needs to be followed by a call to either bdrv_set_perm() or
  * bdrv_abort_perm_update(). */
+__attribute__((unused))
 static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
   uint64_t new_used_perm,
   uint64_t new_shared_perm,
@@ -4100,10 +4103,6 @@ static BlockReopenQueue 
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
 bs_entry->state.explicit_options = explicit_options;
 bs_entry->state.flags = flags;
 
-/* This needs to be overwritten in bdrv_reopen_prepare() */
-bs_entry->state.perm = UINT64_MAX;
-bs_entry->state.shared_perm = 0;
-
 /*
  * If keep_old_opts is false then it means that unspecified
  * options must be reset to their original value. We don't allow
@@ -4186,40 +4185,37 @@ BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue 
*bs_queue,
  */
 int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp)
 {
-int ret = -1;
+int ret = 0;
 BlockReopenQueueEntry *bs_entry, *next;
+GSList *tran = NULL;
+g_autoptr(GHashTable) found = NULL;
+g_autoptr(GSList) refresh_list = NULL;
 
 assert(bs_queue != NULL);
 
 QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
 assert(bs_entry->state.bs->quiesce_counter > 0);
-if (bdrv_reopen_prepare(_entry->state, bs_queue, errp)) {
-goto cleanup;
+ret = bdrv_reopen_prepare(_entry->state, bs_queue, , errp);
+if (ret < 0) {
+goto abort;
 }
 bs_entry->prepared = true;
 }
 
+found = g_hash_table_new(NULL, NULL);
 QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
 BDRVReopenState *state = _entry->state;
-ret = bdrv_check_perm(state->bs, bs_queue, state->perm,
-  state->shared_perm, errp);
-if (ret < 0) {
-goto cleanup_perm;
-}
-/* Check if new_backing_bs would accept the new permissions */
-if (state->replace_backing_bs && state->new_backing_bs) {
-uint64_t nperm, nshared;
-bdrv_child_perm(state->bs, state->new_backing_bs,
-NULL, bdrv_backing_role(state->bs),
-bs_queue, state->perm, state->shared_perm,
-, );
-ret = bdrv_check_update_perm(state->new_backing_bs, NULL,
- nperm, nshared, errp);
-if (ret < 0) {
-goto cleanup_perm;
-}
+
+refresh_list = bdrv_topological_dfs(refresh_list, found, state->bs);
+if (state->old_backing_bs) {
+refresh_list = 

[PATCH v2 36/36] block: refactor bdrv_node_check_perm()

[Vladimir Sementsov-Ogievskiy]

Now, bdrv_node_check_perm() is called only with fresh cumulative
permissions, so its actually "refresh_perm".

Move permission calculation to the function. Also, drop unreachable
error message.

Add also Virtuozzo copyright, as big work is done at this point.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 38 +-
 1 file changed, 9 insertions(+), 29 deletions(-)

diff --git a/block.c b/block.c
index 20b1cf59f7..576b145cbf 100644
--- a/block.c
+++ b/block.c
@@ -2,6 +2,7 @@
  * QEMU System Emulator block driver
  *
  * Copyright (c) 2003 Fabrice Bellard
+ * Copyright (c) 2020 Virtuozzo International GmbH.
  *
  * Permission is hereby granted, free of charge, to any person obtaining a copy
  * of this software and associated documentation files (the "Software"), to 
deal
@@ -2204,23 +2205,15 @@ static void bdrv_replace_child(BdrvChild *child, 
BlockDriverState *new_bs,
 /* old_bs reference is transparently moved from @child to @s */
 }
 
-/*
- * Check whether permissions on this node can be changed in a way that
- * @cumulative_perms and @cumulative_shared_perms are the new cumulative
- * permissions of all its parents. This involves checking whether all necessary
- * permission changes to child nodes can be performed.
- *
- * A call to this function must always be followed by a call to bdrv_set_perm()
- * or bdrv_abort_perm_update().
- */
-static int bdrv_node_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
-uint64_t cumulative_perms,
-uint64_t cumulative_shared_perms,
-GSList **tran, Error **errp)
+static int bdrv_node_refresh_perm(BlockDriverState *bs, BlockReopenQueue *q,
+  GSList **tran, Error **errp)
 {
 BlockDriver *drv = bs->drv;
 BdrvChild *c;
 int ret;
+uint64_t cumulative_perms, cumulative_shared_perms;
+
+bdrv_get_cumulative_perm(bs, _perms, _shared_perms);
 
 /* Write permissions never work with read-only images */
 if ((cumulative_perms & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED)) &&
@@ -2229,15 +,8 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 if (!bdrv_is_writable_after_reopen(bs, NULL)) {
 error_setg(errp, "Block node is read-only");
 } else {
-uint64_t current_perms, current_shared;
-bdrv_get_cumulative_perm(bs, _perms, _shared);
-if (current_perms & (BLK_PERM_WRITE | BLK_PERM_WRITE_UNCHANGED)) {
-error_setg(errp, "Cannot make block node read-only, there is "
-   "a writer on it");
-} else {
-error_setg(errp, "Cannot make block node read-only and create "
-   "a writer on it");
-}
+error_setg(errp, "Cannot make block node read-only, there is "
+   "a writer on it");
 }
 
 return -EPERM;
@@ -2293,7 +2279,6 @@ static int bdrv_list_refresh_perms(GSList *list, 
BlockReopenQueue *q,
GSList **tran, Error **errp)
 {
 int ret;
-uint64_t cumulative_perms, cumulative_shared_perms;
 BlockDriverState *bs;
 
 for ( ; list; list = list->next) {
@@ -2303,12 +2288,7 @@ static int bdrv_list_refresh_perms(GSList *list, 
BlockReopenQueue *q,
 return -EINVAL;
 }
 
-bdrv_get_cumulative_perm(bs, _perms,
- _shared_perms);
-
-ret = bdrv_node_check_perm(bs, q, cumulative_perms,
-   cumulative_shared_perms,
-   tran, errp);
+ret = bdrv_node_refresh_perm(bs, q, tran, errp);
 if (ret < 0) {
 return ret;
 }
-- 
2.21.3

[PATCH v2 35/36] block: rename bdrv_replace_child_safe() to bdrv_replace_child()

[Vladimir Sementsov-Ogievskiy]

We don't have bdrv_replace_child(), so it's time for
bdrv_replace_child_safe() to take its place.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/block.c b/block.c
index 1fde22e4f4..20b1cf59f7 100644
--- a/block.c
+++ b/block.c
@@ -2183,11 +2183,11 @@ static TransactionActionDrv bdrv_replace_child_drv = {
 };
 
 /*
- * bdrv_replace_child_safe
+ * bdrv_replace_child
  *
  * Note: real unref of old_bs is done only on commit.
  */
-static void bdrv_replace_child_safe(BdrvChild *child, BlockDriverState *new_bs,
+static void bdrv_replace_child(BdrvChild *child, BlockDriverState *new_bs,
 GSList **tran)
 {
 BdrvReplaceChildState *s = g_new(BdrvReplaceChildState, 1);
@@ -3040,7 +3040,7 @@ static int bdrv_set_backing_noperm(BlockDriverState *bs,
 }
 
 if (bs->backing && backing_bs) {
-bdrv_replace_child_safe(bs->backing, backing_bs, tran);
+bdrv_replace_child(bs->backing, backing_bs, tran);
 } else if (bs->backing && !backing_bs) {
 bdrv_remove_backing(bs, tran);
 } else if (backing_bs) {
@@ -4679,7 +4679,7 @@ static void bdrv_remove_backing(BlockDriverState *bs, 
GSList **tran)
 }
 
 if (bs->backing->bs) {
-bdrv_replace_child_safe(bs->backing, NULL, tran);
+bdrv_replace_child(bs->backing, NULL, tran);
 }
 
 tran_prepend(tran, _remove_backing_drv, bs->backing);
@@ -4708,7 +4708,7 @@ static int bdrv_replace_node_noperm(BlockDriverState 
*from,
c->name, from->node_name);
 return -EPERM;
 }
-bdrv_replace_child_safe(c, to, tran);
+bdrv_replace_child(c, to, tran);
 }
 
 return 0;
-- 
2.21.3

[PATCH v2 04/36] block: bdrv_append(): return status

[Vladimir Sementsov-Ogievskiy]

Return int status to avoid extra error propagation schemes.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h   |  4 ++--
 block.c | 15 ---
 block/commit.c  |  6 ++
 block/mirror.c  |  6 ++
 blockdev.c  |  6 +++---
 tests/test-bdrv-graph-mod.c |  6 +++---
 6 files changed, 20 insertions(+), 23 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index db37a35cee..ee3f5a6cca 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -344,8 +344,8 @@ int bdrv_create(BlockDriver *drv, const char* filename,
 int bdrv_create_file(const char *filename, QemuOpts *opts, Error **errp);
 
 BlockDriverState *bdrv_new(void);
-void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
- Error **errp);
+int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
+Error **errp);
 void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
Error **errp);
 
diff --git a/block.c b/block.c
index 55efef3c9d..916087ee1a 100644
--- a/block.c
+++ b/block.c
@@ -3103,7 +3103,6 @@ static BlockDriverState 
*bdrv_append_temp_snapshot(BlockDriverState *bs,
 int64_t total_size;
 QemuOpts *opts = NULL;
 BlockDriverState *bs_snapshot = NULL;
-Error *local_err = NULL;
 int ret;
 
 /* if snapshot, we create a temporary backing file and open it
@@ -3145,9 +3144,8 @@ static BlockDriverState 
*bdrv_append_temp_snapshot(BlockDriverState *bs,
 goto out;
 }
 
-bdrv_append(bs_snapshot, bs, _err);
-if (local_err) {
-error_propagate(errp, local_err);
+ret = bdrv_append(bs_snapshot, bs, errp);
+if (ret < 0) {
 bs_snapshot = NULL;
 goto out;
 }
@@ -4606,22 +4604,25 @@ void bdrv_replace_node(BlockDriverState *from, 
BlockDriverState *to,
  * Recent update: bdrv_append does NOT eat bs_new reference for now. Drop this
  * comment several moths later.
  */
-void bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
- Error **errp)
+int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
+Error **errp)
 {
 Error *local_err = NULL;
 
 bdrv_set_backing_hd(bs_new, bs_top, _err);
 if (local_err) {
 error_propagate(errp, local_err);
-return;
+return -EPERM;
 }
 
 bdrv_replace_node(bs_top, bs_new, _err);
 if (local_err) {
 error_propagate(errp, local_err);
 bdrv_set_backing_hd(bs_new, NULL, _abort);
+return -EPERM;
 }
+
+return 0;
 }
 
 static void bdrv_delete(BlockDriverState *bs)
diff --git a/block/commit.c b/block/commit.c
index 61924bcf66..b89bb20b75 100644
--- a/block/commit.c
+++ b/block/commit.c
@@ -254,7 +254,6 @@ void commit_start(const char *job_id, BlockDriverState *bs,
 BlockDriverState *iter;
 BlockDriverState *commit_top_bs = NULL;
 BlockDriverState *filtered_base;
-Error *local_err = NULL;
 int64_t base_size, top_size;
 uint64_t base_perms, iter_shared_perms;
 int ret;
@@ -312,11 +311,10 @@ void commit_start(const char *job_id, BlockDriverState 
*bs,
 
 commit_top_bs->total_sectors = top->total_sectors;
 
-bdrv_append(commit_top_bs, top, _err);
+ret = bdrv_append(commit_top_bs, top, errp);
 bdrv_unref(commit_top_bs); /* referenced by new parents or failed */
-if (local_err) {
+if (ret < 0) {
 commit_top_bs = NULL;
-error_propagate(errp, local_err);
 goto fail;
 }
 
diff --git a/block/mirror.c b/block/mirror.c
index 13f7ecc998..c3fbe3e8bd 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -1560,7 +1560,6 @@ static BlockJob *mirror_start_job(
 BlockDriverState *mirror_top_bs;
 bool target_is_backing;
 uint64_t target_perms, target_shared_perms;
-Error *local_err = NULL;
 int ret;
 
 if (granularity == 0) {
@@ -1606,12 +1605,11 @@ static BlockJob *mirror_start_job(
 mirror_top_bs->opaque = bs_opaque;
 
 bdrv_drained_begin(bs);
-bdrv_append(mirror_top_bs, bs, _err);
+ret = bdrv_append(mirror_top_bs, bs, errp);
 bdrv_drained_end(bs);
 
-if (local_err) {
+if (ret < 0) {
 bdrv_unref(mirror_top_bs);
-error_propagate(errp, local_err);
 return NULL;
 }
 
diff --git a/blockdev.c b/blockdev.c
index 96c96f8ba6..2af35d0958 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1432,6 +1432,7 @@ typedef struct ExternalSnapshotState {
 static void external_snapshot_prepare(BlkActionState *common,
   Error **errp)
 {
+int ret;
 int flags = 0;
 QDict *options = NULL;
 Error *local_err = NULL;
@@ -1587,9 +1588,8 @@ static void external_snapshot_prepare(BlkActionState 
*common,
 goto out;
 }
 
-bdrv_append(state->new_bs, state->old_bs, _err);
-if (local_err) {
-error_propagate(errp, local_err);
+ret = 

[PATCH v2 06/36] block: BdrvChildClass: add .get_parent_aio_context handler

[Vladimir Sementsov-Ogievskiy]

Add new handler to get aio context and implement it in all child
classes. Add corresponding public interface to be used soon.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |  3 +++
 include/block/block_int.h |  2 ++
 block.c   | 13 +
 block/block-backend.c |  9 +
 blockjob.c|  8 
 5 files changed, 35 insertions(+)

diff --git a/include/block/block.h b/include/block/block.h
index 550c5a7513..6788ccd25b 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -688,6 +688,9 @@ bool bdrv_can_set_aio_context(BlockDriverState *bs, 
AioContext *ctx,
   GSList **ignore, Error **errp);
 int bdrv_parent_try_set_aio_context(BdrvChild *c, AioContext *ctx,
 Error **errp);
+
+AioContext *bdrv_child_get_parent_aio_context(BdrvChild *c);
+
 int bdrv_probe_blocksizes(BlockDriverState *bs, BlockSizes *bsz);
 int bdrv_probe_geometry(BlockDriverState *bs, HDGeometry *geo);
 
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 9138aaf5ec..943fd855fe 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -772,6 +772,8 @@ struct BdrvChildClass {
 bool (*can_set_aio_ctx)(BdrvChild *child, AioContext *ctx,
 GSList **ignore, Error **errp);
 void (*set_aio_ctx)(BdrvChild *child, AioContext *ctx, GSList **ignore);
+
+AioContext *(*get_parent_aio_context)(BdrvChild *child);
 };
 
 extern const BdrvChildClass child_of_bds;
diff --git a/block.c b/block.c
index 5d925c208d..95d3684d8d 100644
--- a/block.c
+++ b/block.c
@@ -1334,6 +1334,13 @@ static int bdrv_child_cb_update_filename(BdrvChild *c, 
BlockDriverState *base,
 return 0;
 }
 
+static AioContext *bdrv_child_cb_get_parent_aio_context(BdrvChild *c)
+{
+BlockDriverState *bs = c->opaque;
+
+return bdrv_get_aio_context(bs);
+}
+
 const BdrvChildClass child_of_bds = {
 .parent_is_bds   = true,
 .get_parent_desc = bdrv_child_get_parent_desc,
@@ -1347,8 +1354,14 @@ const BdrvChildClass child_of_bds = {
 .can_set_aio_ctx = bdrv_child_cb_can_set_aio_ctx,
 .set_aio_ctx = bdrv_child_cb_set_aio_ctx,
 .update_filename = bdrv_child_cb_update_filename,
+.get_parent_aio_context = bdrv_child_cb_get_parent_aio_context,
 };
 
+AioContext *bdrv_child_get_parent_aio_context(BdrvChild *c)
+{
+return c->klass->get_parent_aio_context(c);
+}
+
 static int bdrv_open_flags(BlockDriverState *bs, int flags)
 {
 int open_flags = flags;
diff --git a/block/block-backend.c b/block/block-backend.c
index ce78d30794..28efa0dff3 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -298,6 +298,13 @@ static void blk_root_detach(BdrvChild *child)
 }
 }
 
+static AioContext *blk_root_get_parent_aio_context(BdrvChild *c)
+{
+BlockBackend *blk = c->opaque;
+
+return blk_get_aio_context(blk);
+}
+
 static const BdrvChildClass child_root = {
 .inherit_options= blk_root_inherit_options,
 
@@ -318,6 +325,8 @@ static const BdrvChildClass child_root = {
 
 .can_set_aio_ctx= blk_root_can_set_aio_ctx,
 .set_aio_ctx= blk_root_set_aio_ctx,
+
+.get_parent_aio_context = blk_root_get_parent_aio_context,
 };
 
 /*
diff --git a/blockjob.c b/blockjob.c
index 9d0bed01c2..f671763c2c 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -163,6 +163,13 @@ static void child_job_set_aio_ctx(BdrvChild *c, AioContext 
*ctx,
 job->job.aio_context = ctx;
 }
 
+static AioContext *child_job_get_parent_aio_context(BdrvChild *c)
+{
+BlockJob *job = c->opaque;
+
+return job->job.aio_context;
+}
+
 static const BdrvChildClass child_job = {
 .get_parent_desc= child_job_get_parent_desc,
 .drained_begin  = child_job_drained_begin,
@@ -171,6 +178,7 @@ static const BdrvChildClass child_job = {
 .can_set_aio_ctx= child_job_can_set_aio_ctx,
 .set_aio_ctx= child_job_set_aio_ctx,
 .stay_at_node   = true,
+.get_parent_aio_context = child_job_get_parent_aio_context,
 };
 
 void block_job_remove_all_bdrv(BlockJob *job)
-- 
2.21.3

[PATCH v2 01/36] tests/test-bdrv-graph-mod: add test_parallel_exclusive_write

[Vladimir Sementsov-Ogievskiy]

Add the test that shows that concept of ignore_children is incomplete.
Actually, when we want to update something, ignoring permission of some
existing BdrvChild, we should ignore also the propagated effect of this
child to the other children. But that's not done. Better approach
(update permissions on already updated graph) will be implemented
later.

Now the test fails, so it's added with -d argument to not break make
check.

Test fails with

 "Conflicts with use by fl1 as 'backing', which does not allow 'write' on base"

because when updating permissions we can ignore original top->fl1
BdrvChild. But we don't ignore exclusive write permission in fl1->base
BdrvChild, which is propagated. Correct thing to do is make graph
change first and then do permission update from the top node.

To run test do

  ./test-bdrv-graph-mod -d -p /bdrv-graph-mod/parallel-exclusive-write

from /tests.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/test-bdrv-graph-mod.c | 62 +
 1 file changed, 62 insertions(+)

diff --git a/tests/test-bdrv-graph-mod.c b/tests/test-bdrv-graph-mod.c
index 8cff13830e..3b9e6f242f 100644
--- a/tests/test-bdrv-graph-mod.c
+++ b/tests/test-bdrv-graph-mod.c
@@ -44,6 +44,21 @@ static BlockDriver bdrv_no_perm = {
 .bdrv_child_perm = no_perm_default_perms,
 };
 
+static void exclusive_write_perms(BlockDriverState *bs, BdrvChild *c,
+  BdrvChildRole role,
+  BlockReopenQueue *reopen_queue,
+  uint64_t perm, uint64_t shared,
+  uint64_t *nperm, uint64_t *nshared)
+{
+*nperm = BLK_PERM_WRITE;
+*nshared = BLK_PERM_ALL & ~BLK_PERM_WRITE;
+}
+
+static BlockDriver bdrv_exclusive_writer = {
+.format_name = "exclusive-writer",
+.bdrv_child_perm = exclusive_write_perms,
+};
+
 static BlockDriverState *no_perm_node(const char *name)
 {
 return bdrv_new_open_driver(_no_perm, name, BDRV_O_RDWR, 
_abort);
@@ -55,6 +70,12 @@ static BlockDriverState *pass_through_node(const char *name)
 BDRV_O_RDWR, _abort);
 }
 
+static BlockDriverState *exclusive_writer_node(const char *name)
+{
+return bdrv_new_open_driver(_exclusive_writer, name,
+BDRV_O_RDWR, _abort);
+}
+
 /*
  * test_update_perm_tree
  *
@@ -185,8 +206,44 @@ static void test_should_update_child(void)
 blk_unref(root);
 }
 
+/*
+ * test_parallel_exclusive_write
+ *
+ * Check that when we replace node, old permissions of the node being removed
+ * doesn't break the replacement.
+ */
+static void test_parallel_exclusive_write(void)
+{
+BlockDriverState *top = exclusive_writer_node("top");
+BlockDriverState *base = no_perm_node("base");
+BlockDriverState *fl1 = pass_through_node("fl1");
+BlockDriverState *fl2 = pass_through_node("fl2");
+
+bdrv_attach_child(top, fl1, "backing", _of_bds, BDRV_CHILD_DATA,
+  _abort);
+bdrv_attach_child(fl1, base, "backing", _of_bds, BDRV_CHILD_FILTERED,
+  _abort);
+bdrv_attach_child(fl2, base, "backing", _of_bds, BDRV_CHILD_FILTERED,
+  _abort);
+bdrv_ref(base);
+
+bdrv_replace_node(fl1, fl2, _abort);
+
+bdrv_unref(top);
+}
+
 int main(int argc, char *argv[])
 {
+int i;
+bool debug = false;
+
+for (i = 1; i < argc; i++) {
+if (!strcmp(argv[i], "-d")) {
+debug = true;
+break;
+}
+}
+
 bdrv_init();
 qemu_init_main_loop(_abort);
 
@@ -196,5 +253,10 @@ int main(int argc, char *argv[])
 g_test_add_func("/bdrv-graph-mod/should-update-child",
 test_should_update_child);
 
+if (debug) {
+g_test_add_func("/bdrv-graph-mod/parallel-exclusive-write",
+test_parallel_exclusive_write);
+}
+
 return g_test_run();
 }
-- 
2.21.3

[PATCH v2 07/36] block: drop ctx argument from bdrv_root_attach_child

[Vladimir Sementsov-Ogievskiy]

Passing parent aio context is redundant, as child_class and parent
opaque pointer are enough to retrieve it. Drop the argument and use new
bdrv_child_get_parent_aio_context() interface.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block_int.h | 1 -
 block.c   | 8 +---
 block/block-backend.c | 4 ++--
 blockjob.c| 3 +--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/include/block/block_int.h b/include/block/block_int.h
index 943fd855fe..24a04ac2dc 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1278,7 +1278,6 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
   const char *child_name,
   const BdrvChildClass *child_class,
   BdrvChildRole child_role,
-  AioContext *ctx,
   uint64_t perm, uint64_t shared_perm,
   void *opaque, Error **errp);
 void bdrv_root_unref_child(BdrvChild *child);
diff --git a/block.c b/block.c
index 95d3684d8d..15e6ab666e 100644
--- a/block.c
+++ b/block.c
@@ -2640,13 +2640,13 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
   const char *child_name,
   const BdrvChildClass *child_class,
   BdrvChildRole child_role,
-  AioContext *ctx,
   uint64_t perm, uint64_t shared_perm,
   void *opaque, Error **errp)
 {
 BdrvChild *child;
 Error *local_err = NULL;
 int ret;
+AioContext *ctx;
 
 ret = bdrv_check_update_perm(child_bs, NULL, perm, shared_perm, NULL, 
errp);
 if (ret < 0) {
@@ -2666,6 +2666,8 @@ BdrvChild *bdrv_root_attach_child(BlockDriverState 
*child_bs,
 .opaque = opaque,
 };
 
+ctx = bdrv_child_get_parent_aio_context(child);
+
 /* If the AioContexts don't match, first try to move the subtree of
  * child_bs into the AioContext of the new parent. If this doesn't work,
  * try moving the parent into the AioContext of child_bs instead. */
@@ -2721,8 +2723,8 @@ BdrvChild *bdrv_attach_child(BlockDriverState *parent_bs,
 perm, shared_perm, , _perm);
 
 child = bdrv_root_attach_child(child_bs, child_name, child_class,
-   child_role, bdrv_get_aio_context(parent_bs),
-   perm, shared_perm, parent_bs, errp);
+   child_role, perm, shared_perm, parent_bs,
+   errp);
 if (child == NULL) {
 return NULL;
 }
diff --git a/block/block-backend.c b/block/block-backend.c
index 28efa0dff3..357931ee34 100644
--- a/block/block-backend.c
+++ b/block/block-backend.c
@@ -435,7 +435,7 @@ BlockBackend *blk_new_open(const char *filename, const char 
*reference,
 
 blk->root = bdrv_root_attach_child(bs, "root", _root,
BDRV_CHILD_FILTERED | 
BDRV_CHILD_PRIMARY,
-   blk->ctx, perm, BLK_PERM_ALL, blk, 
errp);
+   perm, BLK_PERM_ALL, blk, errp);
 if (!blk->root) {
 blk_unref(blk);
 return NULL;
@@ -849,7 +849,7 @@ int blk_insert_bs(BlockBackend *blk, BlockDriverState *bs, 
Error **errp)
 bdrv_ref(bs);
 blk->root = bdrv_root_attach_child(bs, "root", _root,
BDRV_CHILD_FILTERED | 
BDRV_CHILD_PRIMARY,
-   blk->ctx, blk->perm, blk->shared_perm,
+   blk->perm, blk->shared_perm,
blk, errp);
 if (blk->root == NULL) {
 return -EPERM;
diff --git a/blockjob.c b/blockjob.c
index f671763c2c..01da714755 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -225,8 +225,7 @@ int block_job_add_bdrv(BlockJob *job, const char *name, 
BlockDriverState *bs,
 if (job->job.aio_context != qemu_get_aio_context()) {
 aio_context_release(job->job.aio_context);
 }
-c = bdrv_root_attach_child(bs, name, _job, 0,
-   job->job.aio_context, perm, shared_perm, job,
+c = bdrv_root_attach_child(bs, name, _job, 0, perm, shared_perm, job,
errp);
 if (job->job.aio_context != qemu_get_aio_context()) {
 aio_context_acquire(job->job.aio_context);
-- 
2.21.3

[PATCH v2 00/36] block: update graph permissions update

[Vladimir Sementsov-Ogievskiy]

Hi all!

Here is a proposal of updating graph changing procedures.

The thing brought me here is a question about "activating" filters after
insertion, which is done in mirror_top and backup_top. The problem is
that we can't simply avoid permission conflict when inserting the
filter: during insertion old permissions of relations to be removed
conflicting with new permissions of new created relations. And current
solution is supporting additional "inactive" mode for the filter when it
doesn't require any permissions.

I suggest to change the order of operations: let's first do all graph
relations modifications and then refresh permissions. Of course we'll
need a way to restore old graph if refresh fails.

Another problem with permission update is that we update permissions in
order of DFS which is not always correct. Better is update node when all
its parents already updated and require correct permissions. This needs
a topological sort of nodes prior to permission update, see more in
patches later.

Patches plan:

01,02 - add failing tests to illustrate conceptual problems of current
permission update system.
[Here is side suggestion: we usually add tests after fix, so careful
 reviewer has to change order of patches to check that test fails before
 fix. I add tests in the way the may be simply executed but not yet take
 part in make check. It seems more native: first show the problem, then
 fix it. And when fixed, make tests available for make check]

03-09 - some perparations, refactorings which may go in separate

10 - new transaction API

15 - toplogical sort implemented for permission update, one of new tests
now pass

19 - improve bdrv_replace_node. second new test now pass

26 - drop .active field and activation procedure for backup-top!

30 - update bdrv_reopen_multiple. At this point everything is using new
paradigm of permission update

31-36 - post refactoring, drop old interfaces, we are done.

Note, that this series does nothing with another graph-update problem
discussed under "[PATCH RFC 0/5] Fix accidental crash in iotest 30".

The series based on block-next Max's branch and can be found here:

git: https://src.openvz.org/scm/~vsementsov/qemu.git
tag: up-block-topologic-perm-v2

Vladimir Sementsov-Ogievskiy (36):
  tests/test-bdrv-graph-mod: add test_parallel_exclusive_write
  tests/test-bdrv-graph-mod: add test_parallel_perm_update
  block: bdrv_append(): don't consume reference
  block: bdrv_append(): return status
  block: add bdrv_parent_try_set_aio_context
  block: BdrvChildClass: add .get_parent_aio_context handler
  block: drop ctx argument from bdrv_root_attach_child
  block: make bdrv_reopen_{prepare,commit,abort} private
  block: return value from bdrv_replace_node()
  util: add transactions.c
  block: bdrv_refresh_perms: check parents compliance
  block: refactor bdrv_child* permission functions
  block: rewrite bdrv_child_try_set_perm() using bdrv_refresh_perms()
  block: inline bdrv_child_*() permission functions calls
  block: use topological sort for permission update
  block: add bdrv_drv_set_perm transaction action
  block: add bdrv_list_* permission update functions
  block: add bdrv_replace_child_safe() transaction action
  block: fix bdrv_replace_node_common
  block: add bdrv_attach_child_common() transaction action
  block: add bdrv_attach_child_noperm() transaction action
  block: split out bdrv_replace_node_noperm()
  block: adapt bdrv_append() for inserting filters
  block: add bdrv_remove_backing transaction action
  block: introduce bdrv_drop_filter()
  block/backup-top: drop .active
  block: drop ignore_children for permission update functions
  block: add bdrv_set_backing_noperm() transaction action
  blockdev: qmp_x_blockdev_reopen: acquire all contexts
  block: bdrv_reopen_multiple: refresh permissions on updated graph
  block: drop unused permission update functions
  block: inline bdrv_check_perm_common()
  block: inline bdrv_replace_child()
  block: refactor bdrv_child_set_perm_safe() transaction action
  block: rename bdrv_replace_child_safe() to bdrv_replace_child()
  block: refactor bdrv_node_check_perm()

 include/block/block.h   |   20 +-
 include/block/block_int.h   |8 +-
 include/qemu/transactions.h |   46 ++
 block.c | 1319 ---
 block/backup-top.c  |   39 +-
 block/block-backend.c   |   13 +-
 block/commit.c  |7 +-
 block/file-posix.c  |   84 +--
 block/mirror.c  |9 +-
 blockdev.c  |   33 +-
 blockjob.c  |   11 +-
 tests/test-bdrv-drain.c |2 +-
 tests/test-bdrv-graph-mod.c |  122 +++-
 util/transactions.c |   81 +++
 tests/qemu-iotests/283.out  |2 +-
 util/meson.build|1 +
 16 files changed, 1100 insertions(+), 697 deletions(-)
 create mode 100644 include/qemu/transactions.h
 create mode 100644 util/transactions.c

-- 
2.21.3

[PATCH v2 19/36] block: fix bdrv_replace_node_common

[Vladimir Sementsov-Ogievskiy]

inore_children thing doesn't help to track all propagated permissions
of children we want to ignore. The simplest way to correctly update
permissions is update graph first and then do permission update. In
this case we just referesh permissions for the whole subgraph (in
topological-sort defined order) and everything is correctly calculated
automatically without any ignore_children.

So, refactor bdrv_replace_node_common to first do graph update and then
refresh the permissions.

Test test_parallel_exclusive_write() now pass, so move it out of
debugging "if".

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 42 ++---
 tests/test-bdrv-graph-mod.c | 18 +++-
 2 files changed, 19 insertions(+), 41 deletions(-)

diff --git a/block.c b/block.c
index f24bd60c2f..f0fcd7 100644
--- a/block.c
+++ b/block.c
@@ -2199,7 +2199,6 @@ static TransactionActionDrv bdrv_replace_child_drv = {
  *
  * Note: real unref of old_bs is done only on commit.
  */
-__attribute__((unused))
 static void bdrv_replace_child_safe(BdrvChild *child, BlockDriverState *new_bs,
 GSList **tran)
 {
@@ -4794,8 +4793,9 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
 {
 int ret = -EPERM;
 BdrvChild *c, *next;
-GSList *list = NULL, *p;
-uint64_t perm = 0, shared = BLK_PERM_ALL;
+GSList *tran = NULL;
+g_autoptr(GHashTable) found = NULL;
+g_autoptr(GSList) refresh_list = NULL;
 
 /* Make sure that @from doesn't go away until we have successfully attached
  * all of its parents to @to. */
@@ -4805,7 +4805,12 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
 assert(bdrv_get_aio_context(from) == bdrv_get_aio_context(to));
 bdrv_drained_begin(from);
 
-/* Put all parents into @list and calculate their cumulative permissions */
+/*
+ * Do the replacement without permission update.
+ * Replacement may influence the permissions, we should calculate new
+ * permissions based on new graph. If we fail, we'll roll-back the
+ * replacement.
+ */
 QLIST_FOREACH_SAFE(c, >parents, next_parent, next) {
 assert(c->bs == from);
 if (!should_update_child(c, to)) {
@@ -4821,34 +4826,19 @@ static int bdrv_replace_node_common(BlockDriverState 
*from,
c->name, from->node_name);
 goto out;
 }
-list = g_slist_prepend(list, c);
-perm |= c->perm;
-shared &= c->shared_perm;
-}
-
-/* Check whether the required permissions can be granted on @to, ignoring
- * all BdrvChild in @list so that they can't block themselves. */
-ret = bdrv_check_update_perm(to, NULL, perm, shared, list, errp);
-if (ret < 0) {
-bdrv_abort_perm_update(to);
-goto out;
+bdrv_replace_child_safe(c, to, );
 }
 
-/* Now actually perform the change. We performed the permission check for
- * all elements of @list at once, so set the permissions all at once at the
- * very end. */
-for (p = list; p != NULL; p = p->next) {
-c = p->data;
+found = g_hash_table_new(NULL, NULL);
 
-bdrv_ref(to);
-bdrv_replace_child_noperm(c, to);
-bdrv_unref(from);
-}
+refresh_list = bdrv_topological_dfs(refresh_list, found, to);
+refresh_list = bdrv_topological_dfs(refresh_list, found, from);
 
-bdrv_set_perm(to);
+ret = bdrv_list_refresh_perms(refresh_list, NULL, , errp);
 
 out:
-g_slist_free(list);
+tran_finalize(tran, ret);
+
 bdrv_drained_end(from);
 bdrv_unref(from);
 
diff --git a/tests/test-bdrv-graph-mod.c b/tests/test-bdrv-graph-mod.c
index 0d62e05ddb..93a5941a9b 100644
--- a/tests/test-bdrv-graph-mod.c
+++ b/tests/test-bdrv-graph-mod.c
@@ -294,20 +294,11 @@ static void test_parallel_perm_update(void)
 bdrv_child_refresh_perms(top, top->children.lh_first, _abort);
 
 assert(c_fl1->perm & BLK_PERM_WRITE);
+bdrv_unref(top);
 }
 
 int main(int argc, char *argv[])
 {
-int i;
-bool debug = false;
-
-for (i = 1; i < argc; i++) {
-if (!strcmp(argv[i], "-d")) {
-debug = true;
-break;
-}
-}
-
 bdrv_init();
 qemu_init_main_loop(_abort);
 
@@ -318,11 +309,8 @@ int main(int argc, char *argv[])
 test_should_update_child);
 g_test_add_func("/bdrv-graph-mod/parallel-perm-update",
 test_parallel_perm_update);
-
-if (debug) {
-g_test_add_func("/bdrv-graph-mod/parallel-exclusive-write",
-test_parallel_exclusive_write);
-}
+g_test_add_func("/bdrv-graph-mod/parallel-exclusive-write",
+test_parallel_exclusive_write);
 
 return g_test_run();
 }
-- 
2.21.3

[PATCH v2 31/36] block: drop unused permission update functions

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 103 
 1 file changed, 103 deletions(-)

diff --git a/block.c b/block.c
index 474e624152..3ea04bbd8f 100644
--- a/block.c
+++ b/block.c
@@ -1933,11 +1933,6 @@ static int bdrv_fill_options(QDict **options, const char 
*filename,
 return 0;
 }
 
-static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
-  uint64_t new_used_perm,
-  uint64_t new_shared_perm,
-  Error **errp);
-
 typedef struct BlockReopenQueueEntry {
  bool prepared;
  bool perms_checked;
@@ -2361,56 +2356,12 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 return 0;
 }
 
-static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
-   uint64_t cumulative_perms,
-   uint64_t cumulative_shared_perms, Error **errp)
-{
-g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
-return bdrv_check_perm_common(list, q, true, cumulative_perms,
-  cumulative_shared_perms, NULL, errp);
-}
-
 static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q,
GSList **tran, Error **errp)
 {
 return bdrv_check_perm_common(list, q, false, 0, 0, tran, errp);
 }
 
-/*
- * Notifies drivers that after a previous bdrv_check_perm() call, the
- * permission update is not performed and any preparations made for it (e.g.
- * taken file locks) need to be undone.
- */
-static void bdrv_node_abort_perm_update(BlockDriverState *bs)
-{
-BlockDriver *drv = bs->drv;
-BdrvChild *c;
-
-if (!drv) {
-return;
-}
-
-bdrv_drv_set_perm_abort(bs);
-
-QLIST_FOREACH(c, >children, next) {
-bdrv_child_set_perm_abort(c);
-}
-}
-
-static void bdrv_list_abort_perm_update(GSList *list)
-{
-for ( ; list; list = list->next) {
-bdrv_node_abort_perm_update((BlockDriverState *)list->data);
-}
-}
-
-__attribute__((unused))
-static void bdrv_abort_perm_update(BlockDriverState *bs)
-{
-g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
-return bdrv_list_abort_perm_update(list);
-}
-
 static void bdrv_node_set_perm(BlockDriverState *bs)
 {
 BlockDriver *drv = bs->drv;
@@ -2492,60 +2443,6 @@ char *bdrv_perm_names(uint64_t perm)
 return g_string_free(result, FALSE);
 }
 
-/*
- * Checks whether a new reference to @bs can be added if the new user requires
- * @new_used_perm/@new_shared_perm as its permissions. If @ignore_children is
- * set, the BdrvChild objects in this list are ignored in the calculations;
- * this allows checking permission updates for an existing reference.
- *
- * Needs to be followed by a call to either bdrv_set_perm() or
- * bdrv_abort_perm_update(). */
-__attribute__((unused))
-static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
-  uint64_t new_used_perm,
-  uint64_t new_shared_perm,
-  Error **errp)
-{
-BdrvChild *c;
-uint64_t cumulative_perms = new_used_perm;
-uint64_t cumulative_shared_perms = new_shared_perm;
-
-
-/* There is no reason why anyone couldn't tolerate write_unchanged */
-assert(new_shared_perm & BLK_PERM_WRITE_UNCHANGED);
-
-QLIST_FOREACH(c, >parents, next_parent) {
-if ((new_used_perm & c->shared_perm) != new_used_perm) {
-char *user = bdrv_child_user_desc(c);
-char *perm_names = bdrv_perm_names(new_used_perm & 
~c->shared_perm);
-
-error_setg(errp, "Conflicts with use by %s as '%s', which does not 
"
- "allow '%s' on %s",
-   user, c->name, perm_names, bdrv_get_node_name(c->bs));
-g_free(user);
-g_free(perm_names);
-return -EPERM;
-}
-
-if ((c->perm & new_shared_perm) != c->perm) {
-char *user = bdrv_child_user_desc(c);
-char *perm_names = bdrv_perm_names(c->perm & ~new_shared_perm);
-
-error_setg(errp, "Conflicts with use by %s as '%s', which uses "
- "'%s' on %s",
-   user, c->name, perm_names, bdrv_get_node_name(c->bs));
-g_free(user);
-g_free(perm_names);
-return -EPERM;
-}
-
-cumulative_perms |= c->perm;
-cumulative_shared_perms &= c->shared_perm;
-}
-
-return bdrv_check_perm(bs, q, cumulative_perms, cumulative_shared_perms,
-   errp);
-}
 
 static int bdrv_refresh_perms(BlockDriverState *bs, Error **errp)
 {
-- 
2.21.3

Re: ImageInfo oddities regarding compression

[Kevin Wolf]

Am 27.11.2020 um 13:21 hat Markus Armbruster geschrieben:
> >> I fell down this (thankfully shallow) rabbit hole because we also have
> >> 
> >> { 'enum': 'MultiFDCompression',
> >>   'data': [ 'none', 'zlib',
> >> { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
> >> 
> >> I wonder whether we could merge them into a common type.
> 
> Looks like we could: current code would never report the additional
> value 'none'.  Introspection would show it, though.  Seems unlikely to
> cause trouble.  Observation, not demand.

Forgot to comment on this one...

Technically we could probably, but would it make sense? Support for
compression formats has to be implemented separately for both cases, so
that they currently happen to support the same list is more of a
coincidence.

If we ever add a third compression format to qcow2, would we add the
same format to migration, too, or would we split the schema into two
types again?

Kevin

Re: ImageInfo oddities regarding compression

[Markus Armbruster]

Kevin Wolf  writes:

> Am 27.11.2020 um 13:21 hat Markus Armbruster geschrieben:
>> >> I fell down this (thankfully shallow) rabbit hole because we also have
>> >> 
>> >> { 'enum': 'MultiFDCompression',
>> >>   'data': [ 'none', 'zlib',
>> >> { 'name': 'zstd', 'if': 'defined(CONFIG_ZSTD)' } ] }
>> >> 
>> >> I wonder whether we could merge them into a common type.
>> 
>> Looks like we could: current code would never report the additional
>> value 'none'.  Introspection would show it, though.  Seems unlikely to
>> cause trouble.  Observation, not demand.
>
> Forgot to comment on this one...
>
> Technically we could probably, but would it make sense? Support for
> compression formats has to be implemented separately for both cases, so
> that they currently happen to support the same list is more of a
> coincidence.
>
> If we ever add a third compression format to qcow2, would we add the
> same format to migration, too, or would we split the schema into two
> types again?

I figure if a compression method is worth adding to one, it's probably
worth adding to the other.

Having two separate enums isn't too bad.  A third has been proposed[*],
but I hope we can reuse migration's existing enum there.


[*] [PATCH 1/6] migration: Add multi-thread compress method

[PATCH v3 1/5] monitor: change function obsolete name in comments

[Andrey Shinkevich via]

The function name monitor_qmp_bh_dispatcher() has been changed to
monitor_qmp_dispatcher_co() since the commit 9ce44e2c. Let's amend the
comments.

Signed-off-by: Andrey Shinkevich 
---
 monitor/qmp.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/monitor/qmp.c b/monitor/qmp.c
index b42f8c6..7169366 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
@@ -80,7 +80,7 @@ static void monitor_qmp_cleanup_queue_and_resume(MonitorQMP 
*mon)
 qemu_mutex_lock(>qmp_queue_lock);
 
 /*
- * Same condition as in monitor_qmp_bh_dispatcher(), but before
+ * Same condition as in monitor_qmp_dispatcher_co(), but before
  * removing an element from the queue (hence no `- 1`).
  * Also, the queue should not be empty either, otherwise the
  * monitor hasn't been suspended yet (or was already resumed).
@@ -343,7 +343,7 @@ static void handle_qmp_command(void *opaque, QObject *req, 
Error *err)
 
 /*
  * Suspend the monitor when we can't queue more requests after
- * this one.  Dequeuing in monitor_qmp_bh_dispatcher() or
+ * this one.  Dequeuing in monitor_qmp_dispatcher_co() or
  * monitor_qmp_cleanup_queue_and_resume() will resume it.
  * Note that when OOB is disabled, we queue at most one command,
  * for backward compatibility.
-- 
1.8.3.1

[PATCH v2 17/36] block: add bdrv_list_* permission update functions

[Vladimir Sementsov-Ogievskiy]

Add new interface, allowing use of existing node list. It will be used
to fix bdrv_replace_node() in the further commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 106 +---
 1 file changed, 71 insertions(+), 35 deletions(-)

diff --git a/block.c b/block.c
index 4a43a33401..6996aee1cf 100644
--- a/block.c
+++ b/block.c
@@ -2176,7 +2176,8 @@ static int bdrv_drv_set_perm(BlockDriverState *bs, 
uint64_t perm,
 static int bdrv_node_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
 uint64_t cumulative_perms,
 uint64_t cumulative_shared_perms,
-GSList *ignore_children, Error **errp)
+GSList *ignore_children,
+GSList **tran, Error **errp)
 {
 BlockDriver *drv = bs->drv;
 BdrvChild *c;
@@ -2224,7 +2225,7 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 return 0;
 }
 
-ret = bdrv_drv_set_perm(bs, cumulative_perms, cumulative_shared_perms, 
NULL,
+ret = bdrv_drv_set_perm(bs, cumulative_perms, cumulative_shared_perms, 
tran,
 errp);
 if (ret < 0) {
 return ret;
@@ -2243,36 +2244,53 @@ static int bdrv_node_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 bdrv_child_perm(bs, c->bs, c, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
-bdrv_child_set_perm_safe(c, cur_perm, cur_shared, NULL);
+bdrv_child_set_perm_safe(c, cur_perm, cur_shared, tran);
 }
 
 return 0;
 }
 
-static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
-   uint64_t cumulative_perms,
-   uint64_t cumulative_shared_perms,
-   GSList *ignore_children, Error **errp)
+/*
+ * If use_cumulative_perms is true, use cumulative_perms and
+ * cumulative_shared_perms for first element of the list. Otherwise just 
refresh
+ * all permissions.
+ */
+static int bdrv_check_perm_common(GSList *list, BlockReopenQueue *q,
+  bool use_cumulative_perms,
+  uint64_t cumulative_perms,
+  uint64_t cumulative_shared_perms,
+  GSList *ignore_children,
+  GSList **tran, Error **errp)
 {
 int ret;
-BlockDriverState *root = bs;
-g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, root);
+BlockDriverState *bs;
 
-for ( ; list; list = list->next) {
+if (use_cumulative_perms) {
 bs = list->data;
 
-if (bs != root) {
-if (!bdrv_check_parents_compliance(bs, ignore_children, errp)) {
-return -EINVAL;
-}
+ret = bdrv_node_check_perm(bs, q, cumulative_perms,
+   cumulative_shared_perms,
+   ignore_children, tran, errp);
+if (ret < 0) {
+return ret;
+}
 
-bdrv_get_cumulative_perm(bs, _perms,
- _shared_perms);
+list = list->next;
+}
+
+for ( ; list; list = list->next) {
+bs = list->data;
+
+if (!bdrv_check_parents_compliance(bs, ignore_children, errp)) {
+return -EINVAL;
 }
 
+bdrv_get_cumulative_perm(bs, _perms,
+ _shared_perms);
+
 ret = bdrv_node_check_perm(bs, q, cumulative_perms,
cumulative_shared_perms,
-   ignore_children, errp);
+   ignore_children, tran, errp);
 if (ret < 0) {
 return ret;
 }
@@ -2281,6 +2299,23 @@ static int bdrv_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 return 0;
 }
 
+static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
+   uint64_t cumulative_perms,
+   uint64_t cumulative_shared_perms,
+   GSList *ignore_children, Error **errp)
+{
+g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
+return bdrv_check_perm_common(list, q, true, cumulative_perms,
+  cumulative_shared_perms, ignore_children,
+  NULL, errp);
+}
+
+static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q,
+   GSList **tran, Error **errp)
+{
+return bdrv_check_perm_common(list, q, false, 0, 0, NULL, tran, errp);
+}
+
 /*
  * Notifies drivers that after a previous bdrv_check_perm() call, the
  * permission update is not performed and any preparations made for it (e.g.
@@ -2302,15 +2337,19 @@ static void 

[PATCH v2 09/36] block: return value from bdrv_replace_node()

[Vladimir Sementsov-Ogievskiy]

Functions with errp argument are not recommened to be void-functions.
Improve bdrv_replace_node().

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |  4 ++--
 block.c   | 14 --
 2 files changed, 10 insertions(+), 8 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 5d59984ad4..8f6100dad7 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -346,8 +346,8 @@ int bdrv_create_file(const char *filename, QemuOpts *opts, 
Error **errp);
 BlockDriverState *bdrv_new(void);
 int bdrv_append(BlockDriverState *bs_new, BlockDriverState *bs_top,
 Error **errp);
-void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
-   Error **errp);
+int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
+  Error **errp);
 
 int bdrv_parse_aio(const char *mode, int *flags);
 int bdrv_parse_cache_mode(const char *mode, int *flags, bool *writethrough);
diff --git a/block.c b/block.c
index 3765c7caed..29082c6d47 100644
--- a/block.c
+++ b/block.c
@@ -4537,14 +4537,14 @@ static bool should_update_child(BdrvChild *c, 
BlockDriverState *to)
  * With auto_skip=false the error is returned if from has a parent which should
  * not be updated.
  */
-static void bdrv_replace_node_common(BlockDriverState *from,
- BlockDriverState *to,
- bool auto_skip, Error **errp)
+static int bdrv_replace_node_common(BlockDriverState *from,
+BlockDriverState *to,
+bool auto_skip, Error **errp)
 {
+int ret = -EPERM;
 BdrvChild *c, *next;
 GSList *list = NULL, *p;
 uint64_t perm = 0, shared = BLK_PERM_ALL;
-int ret;
 
 /* Make sure that @from doesn't go away until we have successfully attached
  * all of its parents to @to. */
@@ -4600,10 +4600,12 @@ out:
 g_slist_free(list);
 bdrv_drained_end(from);
 bdrv_unref(from);
+
+return ret;
 }
 
-void bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
-   Error **errp)
+int bdrv_replace_node(BlockDriverState *from, BlockDriverState *to,
+  Error **errp)
 {
 return bdrv_replace_node_common(from, to, true, errp);
 }
-- 
2.21.3

[PATCH v2 14/36] block: inline bdrv_child_*() permission functions calls

[Vladimir Sementsov-Ogievskiy]

Each of them has only one caller. Open-coding simplifies further
pemission-update system changes.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 59 +
 1 file changed, 17 insertions(+), 42 deletions(-)

diff --git a/block.c b/block.c
index 82786ebe1f..92bfcbedc9 100644
--- a/block.c
+++ b/block.c
@@ -1914,11 +1914,11 @@ static int bdrv_fill_options(QDict **options, const 
char *filename,
 return 0;
 }
 
-static int bdrv_child_check_perm(BdrvChild *c, BlockReopenQueue *q,
- uint64_t perm, uint64_t shared,
- GSList *ignore_children, Error **errp);
-static void bdrv_child_abort_perm_update(BdrvChild *c);
-static void bdrv_child_set_perm(BdrvChild *c);
+static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
+  uint64_t new_used_perm,
+  uint64_t new_shared_perm,
+  GSList *ignore_children,
+  Error **errp);
 
 typedef struct BlockReopenQueueEntry {
  bool prepared;
@@ -2166,15 +2166,21 @@ static int bdrv_check_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
 /* Check all children */
 QLIST_FOREACH(c, >children, next) {
 uint64_t cur_perm, cur_shared;
+GSList *cur_ignore_children;
 
 bdrv_child_perm(bs, c->bs, c, c->role, q,
 cumulative_perms, cumulative_shared_perms,
 _perm, _shared);
-ret = bdrv_child_check_perm(c, q, cur_perm, cur_shared, 
ignore_children,
-errp);
+
+cur_ignore_children = g_slist_prepend(g_slist_copy(ignore_children), 
c);
+ret = bdrv_check_update_perm(c->bs, q, cur_perm, cur_shared,
+ cur_ignore_children, errp);
+g_slist_free(cur_ignore_children);
 if (ret < 0) {
 return ret;
 }
+
+bdrv_child_set_perm_safe(c, cur_perm, cur_shared, NULL);
 }
 
 return 0;
@@ -2201,7 +2207,8 @@ static void bdrv_abort_perm_update(BlockDriverState *bs)
 }
 
 QLIST_FOREACH(c, >children, next) {
-bdrv_child_abort_perm_update(c);
+bdrv_child_set_perm_abort(c);
+bdrv_abort_perm_update(c->bs);
 }
 }
 
@@ -2230,7 +2237,8 @@ static void bdrv_set_perm(BlockDriverState *bs)
 
 /* Update all children */
 QLIST_FOREACH(c, >children, next) {
-bdrv_child_set_perm(c);
+bdrv_child_set_perm_commit(c);
+bdrv_set_perm(c->bs);
 }
 }
 
@@ -2338,39 +2346,6 @@ static int bdrv_check_update_perm(BlockDriverState *bs, 
BlockReopenQueue *q,
ignore_children, errp);
 }
 
-/* Needs to be followed by a call to either bdrv_child_set_perm() or
- * bdrv_child_abort_perm_update(). */
-static int bdrv_child_check_perm(BdrvChild *c, BlockReopenQueue *q,
- uint64_t perm, uint64_t shared,
- GSList *ignore_children, Error **errp)
-{
-int ret;
-
-ignore_children = g_slist_prepend(g_slist_copy(ignore_children), c);
-ret = bdrv_check_update_perm(c->bs, q, perm, shared, ignore_children, 
errp);
-g_slist_free(ignore_children);
-
-if (ret < 0) {
-return ret;
-}
-
-bdrv_child_set_perm_safe(c, perm, shared, NULL);
-
-return 0;
-}
-
-static void bdrv_child_set_perm(BdrvChild *c)
-{
-bdrv_child_set_perm_commit(c);
-bdrv_set_perm(c->bs);
-}
-
-static void bdrv_child_abort_perm_update(BdrvChild *c)
-{
-bdrv_child_set_perm_abort(c);
-bdrv_abort_perm_update(c->bs);
-}
-
 static int bdrv_refresh_perms(BlockDriverState *bs, Error **errp)
 {
 int ret;
-- 
2.21.3

[PATCH v2 11/36] block: bdrv_refresh_perms: check parents compliance

[Vladimir Sementsov-Ogievskiy]

Add additional check that node parents do not interfere with each
other. This should not hurt existing callers and allows in further
patch use bdrv_refresh_perms() to update a subtree of changed
BdrvChild (check that change is correct).

New check will substitute bdrv_check_update_perm() in following
permissions refactoring, so keep error messages the same to avoid
unit test result changes.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 63 -
 1 file changed, 54 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 29082c6d47..a756f3e8ad 100644
--- a/block.c
+++ b/block.c
@@ -1966,6 +1966,57 @@ bool bdrv_is_writable(BlockDriverState *bs)
 return bdrv_is_writable_after_reopen(bs, NULL);
 }
 
+static char *bdrv_child_user_desc(BdrvChild *c)
+{
+if (c->klass->get_parent_desc) {
+return c->klass->get_parent_desc(c);
+}
+
+return g_strdup("another user");
+}
+
+static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, Error **errp)
+{
+g_autofree char *user = NULL;
+g_autofree char *perm_names = NULL;
+
+if ((b->perm & a->shared_perm) == b->perm) {
+return true;
+}
+
+perm_names = bdrv_perm_names(b->perm & ~a->shared_perm);
+user = bdrv_child_user_desc(a);
+error_setg(errp, "Conflicts with use by %s as '%s', which does not "
+   "allow '%s' on %s",
+   user, a->name, perm_names, bdrv_get_node_name(b->bs));
+
+return false;
+}
+
+static bool bdrv_check_parents_compliance(BlockDriverState *bs, Error **errp)
+{
+BdrvChild *a, *b;
+
+/*
+ * During the loop we'll look at each pair twice. That's correct is
+ * bdrv_a_allow_b() is asymmetric and we should check each pair in both
+ * directions.
+ */
+QLIST_FOREACH(a, >parents, next_parent) {
+QLIST_FOREACH(b, >parents, next_parent) {
+if (a == b) {
+continue;
+}
+
+if (!bdrv_a_allow_b(a, b, errp)) {
+return false;
+}
+}
+}
+
+return true;
+}
+
 static void bdrv_child_perm(BlockDriverState *bs, BlockDriverState *child_bs,
 BdrvChild *c, BdrvChildRole role,
 BlockReopenQueue *reopen_queue,
@@ -2143,15 +2194,6 @@ void bdrv_get_cumulative_perm(BlockDriverState *bs, 
uint64_t *perm,
 *shared_perm = cumulative_shared_perms;
 }
 
-static char *bdrv_child_user_desc(BdrvChild *c)
-{
-if (c->klass->get_parent_desc) {
-return c->klass->get_parent_desc(c);
-}
-
-return g_strdup("another user");
-}
-
 char *bdrv_perm_names(uint64_t perm)
 {
 struct perm_name {
@@ -2295,6 +2337,9 @@ static int bdrv_refresh_perms(BlockDriverState *bs, Error 
**errp)
 int ret;
 uint64_t perm, shared_perm;
 
+if (!bdrv_check_parents_compliance(bs, errp)) {
+return -EPERM;
+}
 bdrv_get_cumulative_perm(bs, , _perm);
 ret = bdrv_check_perm(bs, NULL, perm, shared_perm, NULL, errp);
 if (ret < 0) {
-- 
2.21.3

[PATCH v2 08/36] block: make bdrv_reopen_{prepare, commit, abort} private

[Vladimir Sementsov-Ogievskiy via]

These functions are called only from bdrv_reopen_multiple() in block.c.
No reason to publish them.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 include/block/block.h |  4 
 block.c   | 13 +
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/include/block/block.h b/include/block/block.h
index 6788ccd25b..5d59984ad4 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -373,10 +373,6 @@ BlockReopenQueue *bdrv_reopen_queue(BlockReopenQueue 
*bs_queue,
 int bdrv_reopen_multiple(BlockReopenQueue *bs_queue, Error **errp);
 int bdrv_reopen_set_read_only(BlockDriverState *bs, bool read_only,
   Error **errp);
-int bdrv_reopen_prepare(BDRVReopenState *reopen_state,
-BlockReopenQueue *queue, Error **errp);
-void bdrv_reopen_commit(BDRVReopenState *reopen_state);
-void bdrv_reopen_abort(BDRVReopenState *reopen_state);
 int bdrv_pwrite_zeroes(BdrvChild *child, int64_t offset,
int bytes, BdrvRequestFlags flags);
 int bdrv_make_zero(BdrvChild *child, BdrvRequestFlags flags);
diff --git a/block.c b/block.c
index 15e6ab666e..3765c7caed 100644
--- a/block.c
+++ b/block.c
@@ -84,6 +84,11 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, AioContext *ctx,
GSList **ignore);
 
+static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue
+   *queue, Error **errp);
+static void bdrv_reopen_commit(BDRVReopenState *reopen_state);
+static void bdrv_reopen_abort(BDRVReopenState *reopen_state);
+
 /* If non-zero, use only whitelisted block drivers */
 static int use_bdrv_whitelist;
 
@@ -4082,8 +4087,8 @@ static int bdrv_reopen_parse_backing(BDRVReopenState 
*reopen_state,
  * commit() for any other BDS that have been left in a prepare() state
  *
  */
-int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue *queue,
-Error **errp)
+static int bdrv_reopen_prepare(BDRVReopenState *reopen_state,
+   BlockReopenQueue *queue, Error **errp)
 {
 int ret = -1;
 int old_flags;
@@ -4298,7 +4303,7 @@ error:
  * makes them final by swapping the staging BlockDriverState contents into
  * the active BlockDriverState contents.
  */
-void bdrv_reopen_commit(BDRVReopenState *reopen_state)
+static void bdrv_reopen_commit(BDRVReopenState *reopen_state)
 {
 BlockDriver *drv;
 BlockDriverState *bs;
@@ -4358,7 +4363,7 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state)
  * Abort the reopen, and delete and free the staged changes in
  * reopen_state
  */
-void bdrv_reopen_abort(BDRVReopenState *reopen_state)
+static void bdrv_reopen_abort(BDRVReopenState *reopen_state)
 {
 BlockDriver *drv;
 
-- 
2.21.3

[PATCH v2 18/36] block: add bdrv_replace_child_safe() transaction action

[Vladimir Sementsov-Ogievskiy]

To be used in the following commit.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 53 +
 1 file changed, 53 insertions(+)

diff --git a/block.c b/block.c
index 6996aee1cf..f24bd60c2f 100644
--- a/block.c
+++ b/block.c
@@ -84,6 +84,8 @@ static BlockDriverState *bdrv_open_inherit(const char 
*filename,
 
 static void bdrv_parent_set_aio_context_ignore(BdrvChild *c, AioContext *ctx,
GSList **ignore);
+static void bdrv_replace_child_noperm(BdrvChild *child,
+  BlockDriverState *new_bs);
 
 static int bdrv_reopen_prepare(BDRVReopenState *reopen_state, BlockReopenQueue
*queue, Error **errp);
@@ -2164,6 +2166,57 @@ static int bdrv_drv_set_perm(BlockDriverState *bs, 
uint64_t perm,
 return 0;
 }
 
+typedef struct BdrvReplaceChildState {
+BdrvChild *child;
+BlockDriverState *old_bs;
+} BdrvReplaceChildState;
+
+static void bdrv_replace_child_commit(void *opaque)
+{
+BdrvReplaceChildState *s = opaque;
+
+bdrv_unref(s->old_bs);
+}
+
+static void bdrv_replace_child_abort(void *opaque)
+{
+BdrvReplaceChildState *s = opaque;
+BlockDriverState *new_bs = s->child->bs;
+
+/* old_bs reference is transparently moved from @s to @s->child */
+bdrv_replace_child_noperm(s->child, s->old_bs);
+bdrv_unref(new_bs);
+}
+
+static TransactionActionDrv bdrv_replace_child_drv = {
+.commit = bdrv_replace_child_commit,
+.abort = bdrv_replace_child_abort,
+.clean = g_free,
+};
+
+/*
+ * bdrv_replace_child_safe
+ *
+ * Note: real unref of old_bs is done only on commit.
+ */
+__attribute__((unused))
+static void bdrv_replace_child_safe(BdrvChild *child, BlockDriverState *new_bs,
+GSList **tran)
+{
+BdrvReplaceChildState *s = g_new(BdrvReplaceChildState, 1);
+*s = (BdrvReplaceChildState) {
+.child = child,
+.old_bs = child->bs,
+};
+tran_prepend(tran, _replace_child_drv, s);
+
+if (new_bs) {
+bdrv_ref(new_bs);
+}
+bdrv_replace_child_noperm(child, new_bs);
+/* old_bs reference is transparently moved from @child to @s */
+}
+
 /*
  * Check whether permissions on this node can be changed in a way that
  * @cumulative_perms and @cumulative_shared_perms are the new cumulative
-- 
2.21.3

[PATCH v2 24/36] block: add bdrv_remove_backing transaction action

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 42 --
 1 file changed, 40 insertions(+), 2 deletions(-)

diff --git a/block.c b/block.c
index 7094922509..b1394b721c 100644
--- a/block.c
+++ b/block.c
@@ -2973,12 +2973,19 @@ out:
 return child;
 }
 
+static void bdrv_child_free(void *opaque)
+{
+BdrvChild *c = opaque;
+
+g_free(c->name);
+g_free(c);
+}
+
 static void bdrv_remove_empty_child(BdrvChild *child)
 {
 assert(!child->bs);
 QLIST_SAFE_REMOVE(child, next);
-g_free(child->name);
-g_free(child);
+bdrv_child_free(child);
 }
 
 typedef struct BdrvAttachChildCommonState {
@@ -4897,6 +4904,37 @@ static bool should_update_child(BdrvChild *c, 
BlockDriverState *to)
 return ret;
 }
 
+/* this doesn't restore original child bs, only the child itself */
+static void bdrv_remove_backing_abort(void *opaque)
+{
+BdrvChild *c = opaque;
+BlockDriverState *parent_bs = c->opaque;
+
+QLIST_INSERT_HEAD(_bs->children, c, next);
+parent_bs->backing = c;
+}
+
+static TransactionActionDrv bdrv_remove_backing_drv = {
+.abort = bdrv_remove_backing_abort,
+.commit = bdrv_child_free,
+};
+
+__attribute__((unused))
+static void bdrv_remove_backing(BlockDriverState *bs, GSList **tran)
+{
+if (!bs->backing) {
+return;
+}
+
+if (bs->backing->bs) {
+bdrv_replace_child_safe(bs->backing, NULL, tran);
+}
+
+tran_prepend(tran, _remove_backing_drv, bs->backing);
+QLIST_SAFE_REMOVE(bs->backing, next);
+bs->backing = NULL;
+}
+
 static int bdrv_replace_node_noperm(BlockDriverState *from,
 BlockDriverState *to,
 bool auto_skip, GSList **tran, Error 
**errp)
-- 
2.21.3

[PATCH v2 27/36] block: drop ignore_children for permission update functions

[Vladimir Sementsov-Ogievskiy]

This argument is always NULL. Drop it.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block.c | 36 +++-
 1 file changed, 11 insertions(+), 25 deletions(-)

diff --git a/block.c b/block.c
index e835a78f06..54fb6d24bd 100644
--- a/block.c
+++ b/block.c
@@ -1934,7 +1934,6 @@ static int bdrv_fill_options(QDict **options, const char 
*filename,
 static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
   uint64_t new_used_perm,
   uint64_t new_shared_perm,
-  GSList *ignore_children,
   Error **errp);
 
 typedef struct BlockReopenQueueEntry {
@@ -2011,9 +2010,7 @@ static bool bdrv_a_allow_b(BdrvChild *a, BdrvChild *b, 
Error **errp)
 return false;
 }
 
-static bool bdrv_check_parents_compliance(BlockDriverState *bs,
-  GSList *ignore_children,
-  Error **errp)
+static bool bdrv_check_parents_compliance(BlockDriverState *bs, Error **errp)
 {
 BdrvChild *a, *b;
 
@@ -2024,9 +2021,7 @@ static bool 
bdrv_check_parents_compliance(BlockDriverState *bs,
  */
 QLIST_FOREACH(a, >parents, next_parent) {
 QLIST_FOREACH(b, >parents, next_parent) {
-if (a == b || g_slist_find(ignore_children, a) ||
-g_slist_find(ignore_children, b))
-{
+if (a == b) {
 continue;
 }
 
@@ -2243,7 +2238,6 @@ static void bdrv_replace_child_safe(BdrvChild *child, 
BlockDriverState *new_bs,
 static int bdrv_node_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
 uint64_t cumulative_perms,
 uint64_t cumulative_shared_perms,
-GSList *ignore_children,
 GSList **tran, Error **errp)
 {
 BlockDriver *drv = bs->drv;
@@ -2326,7 +2320,6 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
   bool use_cumulative_perms,
   uint64_t cumulative_perms,
   uint64_t cumulative_shared_perms,
-  GSList *ignore_children,
   GSList **tran, Error **errp)
 {
 int ret;
@@ -2337,7 +2330,7 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 
 ret = bdrv_node_check_perm(bs, q, cumulative_perms,
cumulative_shared_perms,
-   ignore_children, tran, errp);
+   tran, errp);
 if (ret < 0) {
 return ret;
 }
@@ -2348,7 +2341,7 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 for ( ; list; list = list->next) {
 bs = list->data;
 
-if (!bdrv_check_parents_compliance(bs, ignore_children, errp)) {
+if (!bdrv_check_parents_compliance(bs, errp)) {
 return -EINVAL;
 }
 
@@ -2357,7 +2350,7 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 
 ret = bdrv_node_check_perm(bs, q, cumulative_perms,
cumulative_shared_perms,
-   ignore_children, tran, errp);
+   tran, errp);
 if (ret < 0) {
 return ret;
 }
@@ -2368,19 +2361,17 @@ static int bdrv_check_perm_common(GSList *list, 
BlockReopenQueue *q,
 
 static int bdrv_check_perm(BlockDriverState *bs, BlockReopenQueue *q,
uint64_t cumulative_perms,
-   uint64_t cumulative_shared_perms,
-   GSList *ignore_children, Error **errp)
+   uint64_t cumulative_shared_perms, Error **errp)
 {
 g_autoptr(GSList) list = bdrv_topological_dfs(NULL, NULL, bs);
 return bdrv_check_perm_common(list, q, true, cumulative_perms,
-  cumulative_shared_perms, ignore_children,
-  NULL, errp);
+  cumulative_shared_perms, NULL, errp);
 }
 
 static int bdrv_list_refresh_perms(GSList *list, BlockReopenQueue *q,
GSList **tran, Error **errp)
 {
-return bdrv_check_perm_common(list, q, false, 0, 0, NULL, tran, errp);
+return bdrv_check_perm_common(list, q, false, 0, 0, tran, errp);
 }
 
 /*
@@ -2509,7 +2500,6 @@ char *bdrv_perm_names(uint64_t perm)
 static int bdrv_check_update_perm(BlockDriverState *bs, BlockReopenQueue *q,
   uint64_t new_used_perm,
   uint64_t new_shared_perm,
-  GSList *ignore_children,
   Error **errp)
 {
 BdrvChild *c;
@@