Re: [libvirt] [Qemu-devel] NBD TLS support in QEMU
On 09/04/2014 10:34 AM, Daniel P. Berrange wrote: On Thu, Sep 04, 2014 at 04:19:17PM +0200, Benoît Canet wrote: The Wednesday 03 Sep 2014 à 17:44:17 (+0100), Stefan Hajnoczi wrote : Hi, QEMU offers both NBD client and server functionality. The NBD protocol runs unencrypted, which is a problem when the client and server communicate over an untrusted network. The particular use case that prompted this mail is storage migration in OpenStack. The goal is to encrypt the NBD connection between source and destination hosts during storage migration. I agree this would be usefull. I think we can integrate TLS into the NBD protocol as an optional flag. A quick web search does not reveal existing open source SSL/TLS NBD implementations. I do see a VMware NBDSSL protocol but there is no specification so I guess it is proprietary. The NBD protocol starts with a negotiation phase. This would be the appropriate place to indicate that TLS will be used. After client and server complete TLS setup the connection can continue as normal. Prenegociating TLS look like we will accidentaly introduce some security hole. Why not just using a dedicated port and let the TLS handshake happen normaly ? The mgmt app (libvirt in this case) chooses an arbitrary port when telling QEMU to setup NBD, so we don't need to specify any alternate port. I'd expect that libvirt just tell QEMU to enable NBD at both ends, and we immediately do the TLS handshake upon opening the connection. Only once TLS is established, should the NBD protocol start running. IOW we don't need to modify the NBD protocol at all. This is my understanding of how, for example, the IRC protocol added SSL support. the SSL/TLS handshake happens first, but the very next thing the client/server expects to see is the usual IRC protocol talk, encrypted. If it sees incorrect magic after the SSL shake, both ends hang up. If it sees IRC magic prior to the SSL shake, it either allows an unencrypted session, or if the user or server has requested SSL-only, one or both ends hang up. If the mgmt app tells QEMU to enable TLS at one end and not the other, the mgmt app gets what it deserves (a failed TLS handshake). We certainly would not want QEMU to auto-negotiate and fallback to plain text in this case. Regards, Daniel -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] RFC backup API
On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote:
> On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
>> Hi all,
>>
>> It's been already quite a long time since qemu implemented QMP
>> "drive-backup" command to create block devices backups. Even more,
>> since qemu 2.4 there is a possibility to create incremental backups.
>> Though it is possible to backup all attached to a domain disk drives
>> by combining them into a single QMP transaction command, this way of
>> creating them, not to mention managing, remains inconvenient for an
>> end user of libvirt. Moreover, creating a single drive backup via QMP
>> interface isn't handy either. That said, it looks reasonable to
>> introduce a *new backup API* based on QMP "drive-backup" facilities.
>
> There's also the 'blockdev-backup' command, which seems similar in
> operation to 'drive-backup', but differs subtly.
>
> Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts
> target ID; while 'drive-backup' accept target drive name, otherwise,
> their operation look almost identical, and both commands use
> backup_start() (from qemu/blockdev.c). [Added John Snow in CC to
> correct me if I'm wrong.]
>
No, you're right. Blockdev-backup can backup to an arbitrary device
(which can be backed by a new file), but drive-backup will only accept a
new file.
I don't think blockdev-backup supports incremental backups just yet, but
I don't think there's any reason it can't. (Looking at it: yeah, why
have I not done that yet?...)
> For 'blockdev-backup'
> -
>
> -> { "execute": "blockdev-backup", "arguments": { "device": "src-id",
> "sync": "full",
> "target": "tgt-id" } }
> <- { "return": {} }
>
>
> Where 'tagert' in this case means:
>
> "the name of the backup target device. (json-string)"
>
>
> For 'drive-backup'
> -
>
> -> { "execute": "drive-backup", "arguments": { "device": "drive0",
>"sync": "full",
>"target": "backup.img" } }
> <- { "return": {} }
>
> Here, 'target' means:
>
> "the target of the new image. If the file exists, or if it is a
> device, the existing file/device will be used as the new
> destination. If it does not exist, a new file will be created.
> (json-string)"
>
> [...]
>
>
--js
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] RFC backup API
On 03/25/2016 05:34 AM, Maxim Nestratov wrote:
> 23.03.2016 17:27, John Snow пишет:
>>
>> On 03/23/2016 06:36 AM, Kashyap Chamarthy wrote:
>>> On Mon, Mar 21, 2016 at 01:18:19PM +0300, Maxim Nestratov wrote:
>>>> Hi all,
>>>>
>>>> It's been already quite a long time since qemu implemented QMP
>>>> "drive-backup" command to create block devices backups. Even more,
>>>> since qemu 2.4 there is a possibility to create incremental backups.
>>>> Though it is possible to backup all attached to a domain disk drives
>>>> by combining them into a single QMP transaction command, this way of
>>>> creating them, not to mention managing, remains inconvenient for an
>>>> end user of libvirt. Moreover, creating a single drive backup via QMP
>>>> interface isn't handy either. That said, it looks reasonable to
>>>> introduce a *new backup API* based on QMP "drive-backup" facilities.
>>> There's also the 'blockdev-backup' command, which seems similar in
>>> operation to 'drive-backup', but differs subtly.
>>>
>>> Looking at qmp-commands.hx, I learn that 'blockdev-backup' accepts
>>> target ID; while 'drive-backup' accept target drive name, otherwise,
>>> their operation look almost identical, and both commands use
>>> backup_start() (from qemu/blockdev.c). [Added John Snow in CC to
>>> correct me if I'm wrong.]
>>>
>> No, you're right. Blockdev-backup can backup to an arbitrary device
>> (which can be backed by a new file), but drive-backup will only accept a
>> new file.
>>
>> I don't think blockdev-backup supports incremental backups just yet, but
>> I don't think there's any reason it can't. (Looking at it: yeah, why
>> have I not done that yet?...)
>
> John,
> Any chances to get this implemented? Just not to use two different
> commands in libvirt and start using 'blockdev-backup' right away for
> both full and incremental backups?
>
Yes, it'd be very trivial to do... but I think there is some debate
currently about how to change how incrementals work, so it might not be
wise for me to do this until I know what the situation is, to avoid
having to retcon _two_ QMP interfaces instead of just the one.
(The debate revolves around how bitmaps exist as an in-memory object and
how to present them to the user -- as objects that attach to /drives/,
or to /nodes/, or to both. We haven't answered this question for
ourselves yet, which precludes API design.)
>>> For 'blockdev-backup'
>>> -
>>>
>>> -> { "execute": "blockdev-backup", "arguments": { "device": "src-id",
>>>"sync": "full",
>>>"target": "tgt-id"
>>> } }
>>> <- { "return": {} }
>>>
>>>
>>> Where 'tagert' in this case means:
>>>
>>> "the name of the backup target device. (json-string)"
>>>
>>>
>>> For 'drive-backup'
>>> -
>>>
>>> -> { "execute": "drive-backup", "arguments": { "device": "drive0",
>>> "sync": "full",
>>> "target":
>>> "backup.img" } }
>>> <- { "return": {} }
>>>
>>> Here, 'target' means:
>>>
>>> "the target of the new image. If the file exists, or if it is a
>>> device, the existing file/device will be used as the new
>>> destination. If it does not exist, a new file will be created.
>>> (json-string)"
>>>
>>> [...]
>>>
>>>
>> --js
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 00/20] Incremental Backup API additions
On 10/25/2018 03:20 PM, Eric Blake wrote:
> The following is the latest version of my API proposal for
> incremental backups, and matches the demo I will be presenting
> as part of my KVM Forum 2018 talk tomorrow afternoon:
> https://kvmforum2018.sched.com/event/FzuB/facilitating-incremental-backup-eric-blake-red-hat
>
> The patches are also available via https://repo.or.cz/libvirt/ericb.git
> at the tag backup-v3.
>
> The qemu integration is still incomplete (I don't have push
> mode backups working, only pull mode; and there are still some
> things to fine-tune as I figure out how to better support
> external snapshots and block commit/stream), and I still haven't
> gotten any opinions on whether it is better to have:
>
> virDomainSnapshotCreateXML("...") # existing
> virDomainBackupBegin("...", "...") # this
> series
> virDomainSnapshotCheckpointCreateXML("...",
> "...") # new
>
> or to make checkpoint creation part of the snapshot and backup XML, as in:
>
> virDomainSnapshotCreateXML("...")
> # extension of existing
> virDomainBackupBegin("...") #
> tweak to this series
>
> It's also obvious that we don't want to check in the qemu implementation
> until after Peter's blockdev work has landed AND qemu has gotten rid
> of the x- prefix in its QMP interactions, although the state of this
> series SHOULD be sufficient to prove that the proposed interface is
> at least flexible enough to adapt to whatever qemu tweaks land and
> whatever other hypervisors may support.
For what it's worth, the x prefixes for bitmap manipulation were only
because there was no user for those features; if the overall design of
this API proposal looks palatable to the libvirt community, I'll happily
drop the prefix for any commands actually used by this patchset, before
it's merged.
(From the looks of it, it looks like all of them.)
--js
> Whether to accept the API now> (to make backporting qemu support downstream
> across versions without
> rebasing to an even later date for the API) is a harder question;
> Jirka has already expressed distaste at the idea.
>
> There are probably still loads of things needing fixing in this version,
> I already know I did not address all of Peter's review comments on v2.
> My unit test coverage is not very complete on the XML parsing, and there
> are definitely some hacks in place to get things to the point of a working
> demo that need to be polished for robustness.
>
> Below is a diff since the v2 posting (patches 1-10 here, for later
> patches a comparison against the state of my tree at the time I posted v2):
> https://www.redhat.com/archives/libvir-list/2018-October/msg00700.html
>
> 001/20:[] [--] 'snapshots: Avoid term 'checkpoint' for full system
> snapshot'
> 002/20:[] [--] 'domain_conf: Expose virDomainStorageNetworkParseHost'
> 003/20:[down] 'qemu: Allow optional export name during NBD export'
> 004/20:[] [--] 'backup: Document nuances between different state capture
> APIs'
> 005/20:[] [--] 'backup: Introduce virDomainCheckpointPtr'
> 006/20:[0012] [FC] 'backup: Document new XML for backups'
> 007/20:[] [--] 'backup: Introduce virDomainCheckpoint APIs'
> 008/20:[0024] [FC] 'backup: Introduce virDomainBackup APIs'
> 009/20:[] [--] 'backup: Add new domain:checkpoint access control'
> 010/20:[0001] [FC] 'backup: Implement backup APIs for remote driver'
> 011/20:[0007] [FC] 'wip: backup: Parse and output checkpoint XML'
> 012/20:[0199] [FC] 'wip: backup: Parse and output backup XML'
> 013/20:[] [--] 'backup: Implement virsh support for checkpoints'
> 014/20:[] [--] 'wip: backup: virsh support for backup'
> 015/20:[0223] [FC] 'backup: Add new qemu monitor interactions'
> 016/20:[0123] [FC] 'backup: qemu: Implement metadata tracking for checkpoint
> APIs'
> 017/20:[down] 'wip: backup: Wire up qemu checkpoint commands over QMP'
> 018/20:[0072] [FC] 'wip: backup: qemu: Implement framework for backup job
> APIs'
> 019/20:[down] 'backup: Wire up qemu full pull backup commands over QMP'
> 020/20:[down] 'backup: implement qemu incremental pull backup'
>
> No direct dependency on this one, but included in my git tag:
> https://www.redhat.com/archives/libvir-list/2018-October/msg00899.html
>
> Eric Blake (20):
> snapshots: Avoid term 'checkpoint' for full system snapshot
> domain_conf: Expose virDomainStorageNetworkParseHost
> qemu: Allow optional export name during NBD export
> backup: Document nuances between different state capture APIs
> backup: Introduce virDomainCheckpointPtr
> backup: Document new XML for backups
> backup: Introduce virDomainCheckpoint APIs
> backup: Introduce virDomainBackup APIs
> backup: Add new domain:checkpoint access control
> backup: Implement backup APIs for remote driver
> wip: backup: Parse and output checkpoint XML
> wip: backup: Parse and output backup XML
> backup: Implement virsh support for checkpoints
> wip: backup: virsh support for backup
> backup: Add new qemu monitor
Re: [libvirt] [PATCH 0/8] Work-in-progress: Incremental Backup API additions
CC: Daniel Erez CC: Yaniv Dary CC: Allon Mureinik full thread: https://www.redhat.com/archives/libvir-list/2018-June/msg01066.html On 06/13/2018 12:42 PM, Eric Blake wrote: > I'm offline the rest of this week, but wanted to post the > progress I've made on patches towards the Incremental Backup RFC: > https://www.redhat.com/archives/libvir-list/2018-May/msg01403.html > > Comments welcome, including any naming suggestions > > Still to go: > - Add .rng file for validating the XML format used in virDomainBackupBegin() > - Add flags for validating XML > - Add src/conf/checkpoint_conf.c mirroring src/conf/snapshot_conf.c for > tracking tree of checkpoints > - Add virsh wrappers for calling everything > - Add qemu implementation - my first addition will probably just be for > push model full backups, then additional patches to expand into > pull model (on the qemu list, I still need to review and incorporate > Vladimir's patches for exporting a bitmap over NBD) > - Bug fixes (but why would there be any bugs in the first place? :) > > I've got portions of the qemu code working locally, but not polished > enough to post as a patch yet; my end goal is to have a working demo > against current qemu.git showing the use of virDomainBackupBegin() > for incremental backups with the push model prior to the code freeze > for 4.5.0 this month, even if that code doesn't get checked into > libvirt until later when the qemu code is changed to drop x- prefixes. > (That is, I'm hoping to demo that my API is sound, and thus we can > include the entrypoints in the libvirt.so for this release, even if > the libvirt code for driving pull mode over qemu waits until after a > qemu release where the pieces are promoted to a stable form.) > > Eric Blake (8): > snapshots: Avoid term 'checkpoint' for full system snapshot > backup: Document nuances between different state capture APIs > backup: Introduce virDomainCheckpointPtr > backup: Document new XML for backups > backup: Introduce virDomainCheckpoint APIs > backup: Introduce virDomainBackup APIs > backup: Add new domain:checkpoint access control > backup: Implement backup APIs for remote driver > > docs/Makefile.am| 3 + > docs/apibuild.py| 2 + > docs/docs.html.in | 9 +- > docs/domainstatecapture.html.in | 190 ++ > docs/formatcheckpoint.html.in | 273 + > docs/formatsnapshot.html.in | 16 +- > docs/schemas/domaincheckpoint.rng | 89 +++ > include/libvirt/libvirt-domain-checkpoint.h | 158 + > include/libvirt/libvirt-domain-snapshot.h | 10 +- > include/libvirt/libvirt-domain.h| 14 +- > include/libvirt/libvirt.h | 3 +- > include/libvirt/virterror.h | 5 +- > libvirt.spec.in | 2 + > mingw-libvirt.spec.in | 4 + > po/POTFILES | 1 + > src/Makefile.am | 2 + > src/access/viraccessperm.c | 5 +- > src/access/viraccessperm.h | 8 +- > src/conf/snapshot_conf.c| 2 +- > src/datatypes.c | 62 +- > src/datatypes.h | 31 +- > src/driver-hypervisor.h | 74 ++- > src/libvirt-domain-checkpoint.c | 908 > > src/libvirt-domain-snapshot.c | 4 +- > src/libvirt-domain.c| 8 +- > src/libvirt_private.syms| 2 + > src/libvirt_public.syms | 19 + > src/qemu/qemu_driver.c | 12 +- > src/remote/remote_daemon_dispatch.c | 15 + > src/remote/remote_driver.c | 31 +- > src/remote/remote_protocol.x| 237 +++- > src/remote_protocol-structs | 129 > src/rpc/gendispatch.pl | 32 +- > src/util/virerror.c | 15 +- > tests/domaincheckpointxml2xmlin/empty.xml | 1 + > tests/domaincheckpointxml2xmlout/empty.xml | 10 + > tests/virschematest.c | 2 + > tools/virsh-domain.c| 3 +- > tools/virsh-snapshot.c | 2 +- > tools/virsh.pod | 14 +- > 40 files changed, 2347 insertions(+), 60 deletions(-) > create mode 100644 docs/domainstatecapture.html.in > create mode 100644 docs/formatcheckpoint.html.in > create mode 100644 docs/schemas/domaincheckpoint.rng > create mode 100644 include/libvirt/libvirt-domain-checkpoint.h > create mode 100644 src/libvirt-domain-checkpoint.c > create mode 100644 tests/domaincheckpointxml2xmlin/empty.xml > create mode 100644 tests/domaincheckpointxml2xmlout/empty.xml > -- libvir-list mailing list libvir-list@redhat.com https://www.re
[libvirt] [PATCH v2 1/6] block/dirty-bitmap: add recording and busy properties
The current API allows us to report a single status, which we've defined as:
Frozen: has a successor, treated as qmp_locked, may or may not be enabled.
Locked: no successor, qmp_locked. may or may not be enabled.
Disabled: Not frozen or locked, disabled.
Active: Not frozen, locked, or disabled.
The problem is that both "Frozen" and "Locked" mean nearly the same thing,
and that both of them do not intuit whether they are recording guest writes
or not.
This patch deprecates that status field and introduces two orthogonal
properties instead to replace it.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 9 +
qapi/block-core.json | 10 +-
qemu-deprecated.texi | 6 ++
tests/qemu-iotests/236.out | 28
4 files changed, 52 insertions(+), 1 deletion(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index c6d4acebfa..101383b3af 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -226,6 +226,13 @@ DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap
*bitmap)
}
}
+/* Called with BQL taken. */
+static bool bdrv_dirty_bitmap_recording(BdrvDirtyBitmap *bitmap)
+{
+return !bitmap->disabled || (bitmap->successor &&
+ !bitmap->successor->disabled);
+}
+
/**
* Create a successor bitmap destined to replace this bitmap after an
operation.
* Requires that the bitmap is not frozen and has no successor.
@@ -448,6 +455,8 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
info->has_name = !!bm->name;
info->name = g_strdup(bm->name);
info->status = bdrv_dirty_bitmap_status(bm);
+info->recording = bdrv_dirty_bitmap_recording(bm);
+info->busy = bdrv_dirty_bitmap_user_locked(bm);
info->persistent = bm->persistent;
entry->value = info;
*plist = entry;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 8f23f2ebb8..5d1d182447 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -458,7 +458,14 @@
#
# @granularity: granularity of the dirty bitmap in bytes (since 1.4)
#
-# @status: current status of the dirty bitmap (since 2.4)
+# @status: Deprecated in favor of @recording and @locked. (since 2.4)
+#
+# @recording: true if the bitmap is recording new writes from the guest.
+# Replaces `active` and `disabled` statuses. (since 4.0)
+#
+# @busy: true if the bitmap is in-use by some operation (NBD or jobs)
+#and cannot be modified via QMP or used by another operation.
+#Replaces `locked` and `frozen` statuses. (since 4.0)
#
# @persistent: true if the bitmap will eventually be flushed to persistent
# storage (since 4.0)
@@ -467,6 +474,7 @@
##
{ 'struct': 'BlockDirtyInfo',
'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
+ 'recording': 'bool', 'busy': 'bool',
'status': 'DirtyBitmapStatus', 'persistent': 'bool' } }
##
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 80b0702ad5..f7c9f4c101 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -84,6 +84,12 @@ topologies described with -smp include all possible cpus,
i.e.
"autoload" parameter is now ignored. All bitmaps are automatically loaded
from qcow2 images.
+@subsection query-block dirty-bitmaps.status parameter (since 4.0)
+
+The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
+the query-block command is deprecated. Two new boolean fields,
+``recording'' and ``busy'' effectively replace it.
+
@subsection query-cpus (since 2.12.0)
The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command.
diff --git a/tests/qemu-iotests/236.out b/tests/qemu-iotests/236.out
index 5006f7bca1..815cd053f0 100644
--- a/tests/qemu-iotests/236.out
+++ b/tests/qemu-iotests/236.out
@@ -22,17 +22,21 @@ write -P0xcd 0x3ff 64k
"bitmaps": {
"drive0": [
{
+"busy": false,
"count": 262144,
"granularity": 65536,
"name": "bitmapB",
"persistent": false,
+"recording": true,
"status": "active"
},
{
+"busy": false,
"count": 262144,
"granularity": 65536,
"name": "bitmapA",
"persistent": false,
+"recording": true,
"status": "active"
}
]
@@ -84,17 +88,21 @@ write -P0xcd 0x3ff 64k
"bitmaps": {
"drive0": [
[libvirt] [PATCH v2 0/6] dirty-bitmaps: deprecate @status field
The current internal meanings of "locked", "user_locked", "qmp_locked", "frozen", "enabled", and "disabled" are all a little muddled. Deprecate the @status field in favor of two new booleans that carry very specific meanings. Then, rename and rework some of the internal semantics to help make the API a bit more clear and easier to read. Well, in my opinion. Based on my current bitmaps branch (includes Eric's patch and my documentation update patch.) V2: - All of Eric's suggestions, I hope. - Vladimir's phrasing suggestion on patch 1 - Added a sixth patch that's mostly just motion. I'm on PTO the next two days, so I didn't get to writing a test for the busy bit. I think test 223 is a good candidate, because it uses the NBD functionality. To test with push mode, I need to come up with a blockdebug configuration that will let me pause an incremental backup so I can test race-free. Maybe for V3, sorry. John Snow (6): block/dirty-bitmap: add recording and busy properties block/dirty-bitmaps: rename frozen predicate helper block/dirty-bitmap: change semantics of enabled predicate block/dirty-bitmap: explicitly lock bitmaps with successors block/dirty-bitmaps: unify qmp_locked and user_locked calls block/dirty-bitmaps: move comment block block/dirty-bitmap.c | 110 ++--- blockdev.c | 18 +++--- include/block/dirty-bitmap.h | 7 +-- migration/block-dirty-bitmap.c | 8 +-- nbd/server.c | 6 +- qapi/block-core.json | 10 ++- qemu-deprecated.texi | 6 ++ tests/qemu-iotests/236.out | 28 + 8 files changed, 122 insertions(+), 71 deletions(-) -- 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper
"Frozen" was a good description a long time ago, but it isn't adequate now.
Rename the frozen predicate to has_successor to make the semantics of the
predicate more clear to outside callers.
In the process, remove some calls to frozen() that no longer semantically
make sense. For enabled and disabled in particular, it's actually okay for
the internals to do this but only forbidden for users to invoke them, and
all of the QMP entry uses already check against qmp_locked.
Several other assertions really want to check that the bitmap isn't in-use
by another operation -- use the qmp_locked function for this instead, which
presently also checks for has_successor.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 32 +---
include/block/dirty-bitmap.h | 2 +-
migration/block-dirty-bitmap.c | 2 +-
3 files changed, 19 insertions(+), 17 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 101383b3af..639ebc0645 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -50,7 +50,7 @@ struct BdrvDirtyBitmap {
HBitmap *meta; /* Meta dirty bitmap */
bool qmp_locked;/* Bitmap is locked, it can't be modified
through QMP */
-BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
+BdrvDirtyBitmap *successor; /* Anonymous child; implies user_locked state
*/
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap, in bytes */
bool disabled; /* Bitmap is disabled. It ignores all writes to
@@ -183,14 +183,14 @@ const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap
*bitmap)
}
/* Called with BQL taken. */
-bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap)
+bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap *bitmap)
{
return bitmap->successor;
}
/* Both conditions disallow user-modification via QMP. */
bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_frozen(bitmap) ||
+return bdrv_dirty_bitmap_has_successor(bitmap) ||
bdrv_dirty_bitmap_qmp_locked(bitmap);
}
@@ -215,7 +215,7 @@ bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
/* Called with BQL taken. */
DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap)
{
-if (bdrv_dirty_bitmap_frozen(bitmap)) {
+if (bdrv_dirty_bitmap_has_successor(bitmap)) {
return DIRTY_BITMAP_STATUS_FROZEN;
} else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) {
return DIRTY_BITMAP_STATUS_LOCKED;
@@ -235,7 +235,7 @@ static bool bdrv_dirty_bitmap_recording(BdrvDirtyBitmap
*bitmap)
/**
* Create a successor bitmap destined to replace this bitmap after an
operation.
- * Requires that the bitmap is not frozen and has no successor.
+ * Requires that the bitmap is not locked and has no successor.
* Called with BQL taken.
*/
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
@@ -244,12 +244,16 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState
*bs,
uint64_t granularity;
BdrvDirtyBitmap *child;
-if (bdrv_dirty_bitmap_frozen(bitmap)) {
-error_setg(errp, "Cannot create a successor for a bitmap that is "
- "currently frozen");
+if (bdrv_dirty_bitmap_user_locked(bitmap)) {
+error_setg(errp, "Cannot create a successor for a bitmap that is
in-use "
+ "by an operation");
+return -1;
+}
+if (bdrv_dirty_bitmap_has_successor(bitmap)) {
+error_setg(errp, "Cannot create a successor for a bitmap that already "
+ "has one");
return -1;
}
-assert(!bitmap->successor);
/* Create an anonymous successor */
granularity = bdrv_dirty_bitmap_granularity(bitmap);
@@ -268,7 +272,6 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
-assert(!bdrv_dirty_bitmap_frozen(bitmap));
bitmap->disabled = false;
}
@@ -285,7 +288,7 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap
*bitmap)
static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
assert(!bitmap->active_iterators);
-assert(!bdrv_dirty_bitmap_frozen(bitmap));
+assert(!bdrv_dirty_bitmap_user_locked(bitmap));
assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
@@ -325,7 +328,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
/**
* In cases of failure where we can no longer safely delete the parent,
* we may wish to re-join the parent and child/successor.
- * The merged parent will be un-frozen, but not explicitly re-enabled.
+ * The merged parent will not be user_locked, nor explicitly re-enabled.
* Called withi
[libvirt] [PATCH v2 3/6] block/dirty-bitmap: change semantics of enabled predicate
Currently, enabled means something like "the status of the bitmap
is ACTIVE." After this patch, it should mean exclusively: "This
bitmap is recording guest writes, and is allowed to do so."
In many places, this is how this predicate was already used.
We'll allow users to call user_locked if they're really curious about
finding out if the bitmap is in use by an operation.
To accommodate this, modify the create_successor routine to now
explicitly disable the parent bitmap at creation time.
Justifications:
1. bdrv_dirty_bitmap_status suffers no change from the lack of
1:1 parity with the new predicates because of the order in which
the predicates are checked. This is now only for compatibility.
2. bdrv_set_dirty_bitmap is only used by mirror, which does not use
disabled bitmaps -- all of these writes are internal usages.
Therefore, we should allow writes even in the disabled state.
The condition is removed.
3. bdrv_reset_dirty_bitmap Similarly, this is only used internally by
mirror and migration. In these contexts it is always enabled anyway,
but our API does not need to enforce this.
4. bdrv_set_dirty() is unchanged: pre-patch, it was skipping bitmaps that were
disabled or had a successor, while post-patch it is only skipping bitmaps
that are disabled. To accommodate this, create_successor now ensures that
any bitmap with a successor is explicitly disabled.
5. qcow2_store_persistent_dirty_bitmaps: This only ever wanted to check if the
bitmap was enabled or not. Theoretically if we save during an operation,
this now gets set as enabled instead of disabled, but this cannot happen
in practice because jobs must be finished before we close the disk.
6. block_dirty_bitmap_enable_prepare only ever cared about the
literal bit, and already checked for user_locked beforehand.
7. block_dirty_bitmap_disable_prepare ditto as above.
8. init_dirty_bitmap_migration also already checks user_locked,
so this call can be a simple enabled/disabled check.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
---
block/dirty-bitmap.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 639ebc0645..bb2e19e0d8 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -209,7 +209,7 @@ bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap *bitmap)
/* Called with BQL taken. */
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
{
-return !(bitmap->disabled || bitmap->successor);
+return !bitmap->disabled;
}
/* Called with BQL taken. */
@@ -264,6 +264,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
/* Successor will be on or off based on our current state. */
child->disabled = bitmap->disabled;
+bitmap->disabled = true;
/* Install the successor and freeze the parent */
bitmap->successor = child;
@@ -346,6 +347,8 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
error_setg(errp, "Merging of parent and successor bitmap failed");
return NULL;
}
+
+parent->disabled = successor->disabled;
bdrv_release_dirty_bitmap_locked(successor);
parent->successor = NULL;
@@ -542,7 +545,6 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter)
void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
assert(!bdrv_dirty_bitmap_readonly(bitmap));
hbitmap_set(bitmap->bitmap, offset, bytes);
}
@@ -559,7 +561,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
assert(!bdrv_dirty_bitmap_readonly(bitmap));
hbitmap_reset(bitmap->bitmap, offset, bytes);
}
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2 6/6] block/dirty-bitmaps: move comment block
Simply move the big status enum comment block to above the status
function, and document it as being deprecated. The whole confusing
block can get deleted in three releases time.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 36 +++-
1 file changed, 19 insertions(+), 17 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 8ab048385a..fc390cae94 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -28,22 +28,6 @@
#include "block/block_int.h"
#include "block/blockjob.h"
-/**
- * A BdrvDirtyBitmap can be in four possible user-visible states:
- * (1) Active: successor is NULL, and disabled is false: full r/w mode
- * (2) Disabled: successor is NULL, and disabled is true: qualified r/w mode,
- * guest writes are dropped, but monitor writes are possible,
- * through commands like merge and clear.
- * (3) Frozen: successor is not NULL.
- * A frozen bitmap cannot be renamed, deleted, cleared, set,
- * enabled, merged to, etc. A frozen bitmap can only abdicate()
- * or reclaim().
- * In this state, the anonymous successor bitmap may be either
- * Active and recording writes from the guest (e.g. backup jobs),
- * but it can be Disabled and not recording writes.
- * (4) Locked: Whether Active or Disabled, the user cannot modify this bitmap
- * in any way from the monitor.
- */
struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
@@ -205,7 +189,25 @@ bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
return !bitmap->disabled;
}
-/* Called with BQL taken. */
+/**
+ * bdrv_dirty_bitmap_status: This API is now deprecated.
+ * Called with BQL taken.
+ *
+ * A BdrvDirtyBitmap can be in four possible user-visible states:
+ * (1) Active: successor is NULL, and disabled is false: full r/w mode
+ * (2) Disabled: successor is NULL, and disabled is true: qualified r/w mode,
+ * guest writes are dropped, but monitor writes are possible,
+ * through commands like merge and clear.
+ * (3) Frozen: successor is not NULL.
+ * A frozen bitmap cannot be renamed, deleted, cleared, set,
+ * enabled, merged to, etc. A frozen bitmap can only abdicate()
+ * or reclaim().
+ * In this state, the anonymous successor bitmap may be either
+ * Active and recording writes from the guest (e.g. backup jobs),
+ * but it can be Disabled and not recording writes.
+ * (4) Locked: Whether Active or Disabled, the user cannot modify this bitmap
+ * in any way from the monitor.
+ */
DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap)
{
if (bdrv_dirty_bitmap_has_successor(bitmap)) {
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2 5/6] block/dirty-bitmaps: unify qmp_locked and user_locked calls
These mean the same thing now. Unify them and rename the merged call
bdrv_dirty_bitmap_busy to indicate semantically what we are describing,
as well as help disambiguate from the various _locked and _unlocked
versions of bitmap helpers that refer to mutex locks.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
---
block/dirty-bitmap.c | 41 +++---
blockdev.c | 18 +++
include/block/dirty-bitmap.h | 5 ++---
migration/block-dirty-bitmap.c | 6 ++---
nbd/server.c | 6 ++---
5 files changed, 35 insertions(+), 41 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 2042c62602..8ab048385a 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -48,9 +48,9 @@ struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
HBitmap *meta; /* Meta dirty bitmap */
-bool qmp_locked;/* Bitmap is locked, it can't be modified
- through QMP */
-BdrvDirtyBitmap *successor; /* Anonymous child; implies user_locked state
*/
+bool busy; /* Bitmap is busy, it can't be modified through
+ QMP */
+BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap, in bytes */
bool disabled; /* Bitmap is disabled. It ignores all writes to
@@ -188,22 +188,17 @@ bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap
*bitmap)
return bitmap->successor;
}
-bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_qmp_locked(bitmap);
+bool bdrv_dirty_bitmap_busy(BdrvDirtyBitmap *bitmap) {
+return bitmap->busy;
}
-void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked)
+void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy)
{
qemu_mutex_lock(bitmap->mutex);
-bitmap->qmp_locked = qmp_locked;
+bitmap->busy = busy;
qemu_mutex_unlock(bitmap->mutex);
}
-bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap *bitmap)
-{
-return bitmap->qmp_locked;
-}
-
/* Called with BQL taken. */
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
{
@@ -215,7 +210,7 @@ DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap
*bitmap)
{
if (bdrv_dirty_bitmap_has_successor(bitmap)) {
return DIRTY_BITMAP_STATUS_FROZEN;
-} else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) {
+} else if (bdrv_dirty_bitmap_busy(bitmap)) {
return DIRTY_BITMAP_STATUS_LOCKED;
} else if (!bdrv_dirty_bitmap_enabled(bitmap)) {
return DIRTY_BITMAP_STATUS_DISABLED;
@@ -242,7 +237,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
uint64_t granularity;
BdrvDirtyBitmap *child;
-if (bdrv_dirty_bitmap_user_locked(bitmap)) {
+if (bdrv_dirty_bitmap_busy(bitmap)) {
error_setg(errp, "Cannot create a successor for a bitmap that is
in-use "
"by an operation");
return -1;
@@ -266,7 +261,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
/* Install the successor and lock the parent */
bitmap->successor = child;
-bitmap->qmp_locked = true;
+bitmap->busy = true;
return 0;
}
@@ -288,7 +283,7 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap
*bitmap)
static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
assert(!bitmap->active_iterators);
-assert(!bdrv_dirty_bitmap_user_locked(bitmap));
+assert(!bdrv_dirty_bitmap_busy(bitmap));
assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
@@ -320,7 +315,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
bitmap->successor = NULL;
successor->persistent = bitmap->persistent;
bitmap->persistent = false;
-bitmap->qmp_locked = false;
+bitmap->busy = false;
bdrv_release_dirty_bitmap(bs, bitmap);
return successor;
@@ -329,7 +324,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
/**
* In cases of failure where we can no longer safely delete the parent,
* we may wish to re-join the parent and child/successor.
- * The merged parent will not be user_locked, nor explicitly re-enabled.
+ * The merged parent will not be busy, nor explicitly re-enabled.
* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken.
*/
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
@@ -349,7 +344,7 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
}
parent->disabled = successor->disabled;
-parent->qmp_locked = false;
+parent->busy = false;
bdr
[libvirt] [PATCH v2 4/6] block/dirty-bitmap: explicitly lock bitmaps with successors
Instead of implying a locked status, make it explicit.
Now, bitmaps in use by migration, NBD or backup operations
are all treated the same way with the same code paths.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
---
block/dirty-bitmap.c | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index bb2e19e0d8..2042c62602 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -188,10 +188,8 @@ bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap
*bitmap)
return bitmap->successor;
}
-/* Both conditions disallow user-modification via QMP. */
bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_has_successor(bitmap) ||
- bdrv_dirty_bitmap_qmp_locked(bitmap);
+return bdrv_dirty_bitmap_qmp_locked(bitmap);
}
void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked)
@@ -266,8 +264,9 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
child->disabled = bitmap->disabled;
bitmap->disabled = true;
-/* Install the successor and freeze the parent */
+/* Install the successor and lock the parent */
bitmap->successor = child;
+bitmap->qmp_locked = true;
return 0;
}
@@ -321,6 +320,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
bitmap->successor = NULL;
successor->persistent = bitmap->persistent;
bitmap->persistent = false;
+bitmap->qmp_locked = false;
bdrv_release_dirty_bitmap(bs, bitmap);
return successor;
@@ -349,6 +349,7 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
}
parent->disabled = successor->disabled;
+parent->qmp_locked = false;
bdrv_release_dirty_bitmap_locked(successor);
parent->successor = NULL;
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 6/6] block/dirty-bitmaps: move comment block
On 2/18/19 12:39 PM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> Simply move the big status enum comment block to above the status >> function, and document it as being deprecated. The whole confusing >> block can get deleted in three releases time. >> >> Signed-off-by: John Snow > > > Reviewed-by: Vladimir Sementsov-Ogievskiy > > But I think, it's OK to remove it now. It mostly unrelated to code now, so, > is it needed? And it is unrelated to deprecation. As I understand, user-seen > information is DirtyBitmapStatus declaration in qapi, and it is deprecated > and to be removed in three releases. > I decided to keep it because I wanted to document the particulars of what the fields meant internally before we finally remove it some future release, in case we need to change this function around to maintain backwards compatibility. It'll get removed at that point in time. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper
On 2/18/19 8:57 AM, Vladimir Sementsov-Ogievskiy wrote:
> 14.02.2019 2:23, John Snow wrote:
>> "Frozen" was a good description a long time ago, but it isn't adequate now.
>> Rename the frozen predicate to has_successor to make the semantics of the
>> predicate more clear to outside callers.
>>
>> In the process, remove some calls to frozen() that no longer semantically
>> make sense. For enabled and disabled in particular, it's actually okay for
>> the internals to do this but only forbidden for users to invoke them, and
>
> I'm a bit lost in this paragraph.. to this - what?, to invoke them - whom?
> I think, it would be simpler for me to read patch itself :)
>
Touched this up. I meant enable and disable, not enabled and disabled.
>> all of the QMP entry uses already check against qmp_locked.
>>
>> Several other assertions really want to check that the bitmap isn't in-use
>> by another operation -- use the qmp_locked function for this instead, which
>> presently also checks for has_successor.
>
> hm, you mean user_locked, not qmp_locked.
>
Yes.
[...]
>> /**
>>* Create a successor bitmap destined to replace this bitmap after an
>> operation.
>> - * Requires that the bitmap is not frozen and has no successor.
>> + * Requires that the bitmap is not locked and has no successor.
>
> I think, user_locked, to not interfere with bitmaps mutex. And you use
> user_locked in
> other comments in this patch.
>
You're right. It gets changed again later, but I didn't make this easy
to read.
>>* Called with BQL taken.
>>*/
>> int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
>> @@ -244,12 +244,16 @@ int
>> bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
>> uint64_t granularity;
>> BdrvDirtyBitmap *child;
>>
>> -if (bdrv_dirty_bitmap_frozen(bitmap)) {
>> -error_setg(errp, "Cannot create a successor for a bitmap that is "
>> - "currently frozen");
>> +if (bdrv_dirty_bitmap_user_locked(bitmap)) {
>> +error_setg(errp, "Cannot create a successor for a bitmap that is
>> in-use "
>> + "by an operation");
>> +return -1;
>> +}
>> +if (bdrv_dirty_bitmap_has_successor(bitmap)) {
>> +error_setg(errp, "Cannot create a successor for a bitmap that
>> already "
>> + "has one");
>
>
> Amm, dead code? _user_locked() implies no successor, so we instead can keep
> an assertion..
>
It gets changed later in the series, but I didn't do a great job of
explaining that in advance. I'll amend the commit message to explain
what I'm trying to do.
I tried to hint at this with: "which presently also checks for
has_successor" as an admission that it was redundant, but I need to call
it out in stronger language.
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 3/6] block/dirty-bitmap: change semantics of enabled predicate
On 2/18/19 11:39 AM, Vladimir Sementsov-Ogievskiy wrote:
> 14.02.2019 2:23, John Snow wrote:
>> Currently, enabled means something like "the status of the bitmap
>> is ACTIVE." After this patch, it should mean exclusively: "This
>> bitmap is recording guest writes, and is allowed to do so."
>>
>> In many places, this is how this predicate was already used.
>> We'll allow users to call user_locked if they're really curious about
>> finding out if the bitmap is in use by an operation.
>>
>> To accommodate this, modify the create_successor routine to now
>> explicitly disable the parent bitmap at creation time.
>>
>>
>> Justifications:
>>
>> 1. bdrv_dirty_bitmap_status suffers no change from the lack of
>> 1:1 parity with the new predicates because of the order in which
>> the predicates are checked. This is now only for compatibility.
>>
>> 2. bdrv_set_dirty_bitmap is only used by mirror, which does not use
>> disabled bitmaps -- all of these writes are internal usages.
>> Therefore, we should allow writes even in the disabled state.
>> The condition is removed.
>>
>> 3. bdrv_reset_dirty_bitmap Similarly, this is only used internally by
>> mirror and migration. In these contexts it is always enabled anyway,
>> but our API does not need to enforce this.
>>
>> 4. bdrv_set_dirty() is unchanged: pre-patch, it was skipping bitmaps that
>> were
>> disabled or had a successor, while post-patch it is only skipping bitmaps
>> that are disabled. To accommodate this, create_successor now ensures that
>> any bitmap with a successor is explicitly disabled.
>>
>
> 5-8 are examples of "this is how this predicate was already used"
>
>> 5. qcow2_store_persistent_dirty_bitmaps: This only ever wanted to check if
>> the
>> bitmap was enabled or not. Theoretically if we save during an operation,
>> this now gets set as enabled instead of disabled,
>
> No, as you explicitly disable bitmap in create_successor, so bitmaps with
> successor
> will be disabled anyway.
>
Well, yeah. There's no way it happens in practice currently. It's just
"theoretically" from the viewpoint of the API call itself. There's
nothing stopping a developer from making that call, and this is a
potential change in behavior that we don't expect to observe. Just
noting it down.
> Hmm, and this shows, that actually, you don't need this big description for
> all calls,
> as actually nothing changed and all calls may be described like (4.). Except
> (2. and 3.),
> as these calls are removed (so, is it worth to split them into separate
> previous patch?)
>
I could, to at least have its own justification in a commit message
apart from these -- but at this point it's primarily a benefit for Eric,
You, and myself.
> but this cannot happen
>> in practice because jobs must be finished before we close the disk.
>>
>> 6. block_dirty_bitmap_enable_prepare only ever cared about the
>> literal bit, and already checked for user_locked beforehand.
>>
>> 7. block_dirty_bitmap_disable_prepare ditto as above.
>>
>> 8. init_dirty_bitmap_migration also already checks user_locked,
>> so this call can be a simple enabled/disabled check.
>
>
> hmmm
> 9. nbd_export_new, which too checks bdrv_dirty_bitmap_user_locked but _after_
> call to bdrv_dirty_bitmap_enabled. Anyway it's not changed as described
> in (4.),
> I think it is better to check _user_locked first.
>
You're right, and Eric left a similar feedback elsewhere. user_locked is
the more obvious disqualifier. I think this ought to be its own small
patch because it has nothing much to do with this one.
>
>>
>> Signed-off-by: John Snow
>> Reviewed-by: Eric Blake
>> ---
>> block/dirty-bitmap.c | 7 ---
>> 1 file changed, 4 insertions(+), 3 deletions(-)
>>
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 639ebc0645..bb2e19e0d8 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>> @@ -209,7 +209,7 @@ bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap
>> *bitmap)
>> /* Called with BQL taken. */
>> bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
>> {
>> -return !(bitmap->disabled || bitmap->successor);
>> +return !bitmap->disabled;
>> }
>>
>> /* Called with BQL taken. */
>> @@ -264,6 +264,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState
>> *bs,
>>
>>
Re: [libvirt] [PATCH v2 5/6] block/dirty-bitmaps: unify qmp_locked and user_locked calls
On 2/18/19 12:27 PM, Vladimir Sementsov-Ogievskiy wrote:
> 14.02.2019 2:23, John Snow wrote:
>> These mean the same thing now. Unify them and rename the merged call
>> bdrv_dirty_bitmap_busy to indicate semantically what we are describing,
>> as well as help disambiguate from the various _locked and _unlocked
>> versions of bitmap helpers that refer to mutex locks.
>>
>> Signed-off-by: John Snow
>> Reviewed-by: Eric Blake
>> ---
>> block/dirty-bitmap.c | 41 +++---
>> blockdev.c | 18 +++
>> include/block/dirty-bitmap.h | 5 ++---
>> migration/block-dirty-bitmap.c | 6 ++---
>> nbd/server.c | 6 ++---
>> 5 files changed, 35 insertions(+), 41 deletions(-)
>>
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 2042c62602..8ab048385a 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>> @@ -48,9 +48,9 @@ struct BdrvDirtyBitmap {
>> QemuMutex *mutex;
>> HBitmap *bitmap;/* Dirty bitmap implementation */
>> HBitmap *meta; /* Meta dirty bitmap */
>> -bool qmp_locked;/* Bitmap is locked, it can't be modified
>> - through QMP */
>> -BdrvDirtyBitmap *successor; /* Anonymous child; implies user_locked
>> state */
>> +bool busy; /* Bitmap is busy, it can't be modified
>> through
>> + QMP */
>
> better not "modified" but "used".. for example, export through NBD is not a
> modification.
>
True.
>> +BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
>
> hm this comment change about successor relates more to previous patch, but I
> don't really care.
>
Oh, true.
>> char *name; /* Optional non-empty unique ID */
>> int64_t size; /* Size of the bitmap, in bytes */
>> bool disabled; /* Bitmap is disabled. It ignores all
>> writes to
>> @@ -188,22 +188,17 @@ bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap
>> *bitmap)
>> return bitmap->successor;
>> }
>>
>
>
> In comment for bdrv_dirty_bitmap_create_successor, there is "locked" word,
> which you forget to fix to "busy"
> with at least this fixed:
Good spot. Too many occurrences of the word "lock" to have looked for
them all.
> Reviewed-by: Vladimir Sementsov-Ogievskiy
>
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 4/6] block/dirty-bitmap: explicitly lock bitmaps with successors
On 2/18/19 11:52 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.02.2019 2:23, John Snow wrote: >> Instead of implying a locked status, make it explicit. > > locked interferes with bitmap mutex, so may be better "qmp_locked state", but > not sure. > I agree that "locked" has too many meanings, so in patch 5 I start using the term "busy" instead. >> Now, bitmaps in use by migration, NBD or backup operations >> are all treated the same way with the same code paths. >> >> Signed-off-by: John Snow >> Reviewed-by: Eric Blake > > Reviewed-by: Vladimir Sementsov-Ogievskiy > > Hmm. Isn't it better to make successor-related staff unrelated to locking at > all? Maybe -- but it doesn't make sense to allow users to modify bitmaps that have a successor because we know it's definitely busy. I'll take a further cleanup patch if you think it's better -- just be careful to make sure that any interface calls will work gracefully with a bitmap with a successor. > So, backup will call set_qmp_locked like others? And then do create_successor, > abdicate, reclaim, whatever it wants, and finally set_qmp_locked(false) ? > To make it work even more in the same path. But it may be done separately, if > we > want. > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 2/6] block/dirty-bitmaps: rename frozen predicate helper
On 2/19/19 5:17 AM, Vladimir Sementsov-Ogievskiy wrote:
> 19.02.2019 1:32, John Snow wrote:
>>
>>
>> On 2/18/19 8:57 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> 14.02.2019 2:23, John Snow wrote:
>>>> "Frozen" was a good description a long time ago, but it isn't adequate now.
>>>> Rename the frozen predicate to has_successor to make the semantics of the
>>>> predicate more clear to outside callers.
>>>>
>>>> In the process, remove some calls to frozen() that no longer semantically
>>>> make sense. For enabled and disabled in particular, it's actually okay for
>>>> the internals to do this but only forbidden for users to invoke them, and
>>>
>>> I'm a bit lost in this paragraph.. to this - what?, to invoke them - whom?
>>> I think, it would be simpler for me to read patch itself :)
>>>
>>
>> Touched this up. I meant enable and disable, not enabled and disabled.
>>
>>>> all of the QMP entry uses already check against qmp_locked.
>>>>
>>>> Several other assertions really want to check that the bitmap isn't in-use
>>>> by another operation -- use the qmp_locked function for this instead, which
>>>> presently also checks for has_successor.
>>>
>>> hm, you mean user_locked, not qmp_locked.
>>>
>>
>> Yes.
>>
>> [...]
>>
>>>>/**
>>>> * Create a successor bitmap destined to replace this bitmap after an
>>>> operation.
>>>> - * Requires that the bitmap is not frozen and has no successor.
>>>> + * Requires that the bitmap is not locked and has no successor.
>>>
>>> I think, user_locked, to not interfere with bitmaps mutex. And you use
>>> user_locked in
>>> other comments in this patch.
>>>
>>
>> You're right. It gets changed again later, but I didn't make this easy
>> to read.
>>
>>>> * Called with BQL taken.
>>>> */
>>>>int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
>>>> @@ -244,12 +244,16 @@ int
>>>> bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
>>>>uint64_t granularity;
>>>>BdrvDirtyBitmap *child;
>>>>
>>>> -if (bdrv_dirty_bitmap_frozen(bitmap)) {
>>>> -error_setg(errp, "Cannot create a successor for a bitmap that is "
>>>> - "currently frozen");
>>>> +if (bdrv_dirty_bitmap_user_locked(bitmap)) {
>>>> +error_setg(errp, "Cannot create a successor for a bitmap that is
>>>> in-use "
>>>> + "by an operation");
>>>> +return -1;
>>>> +}
>>>> +if (bdrv_dirty_bitmap_has_successor(bitmap)) {
>>>> +error_setg(errp, "Cannot create a successor for a bitmap that
>>>> already "
>>>> + "has one");
>>>
>>>
>>> Amm, dead code? _user_locked() implies no successor, so we instead can keep
>>> an assertion..
>>>
>>
>> It gets changed later in the series, but I didn't do a great job of
>> explaining that in advance. I'll amend the commit message to explain
>> what I'm trying to do.
>>
>> I tried to hint at this with: "which presently also checks for
>> has_successor" as an admission that it was redundant, but I need to call
>> it out in stronger language.
>>
>
> Hmm, isn't it better to keep an assertion, than add dead code, to be fixed in
> later commits?
>
Eh. I wrote code that looked semantically correct without worrying about
what the calls actually do:
- We want to make sure this bitmap is not in use (user_locked,
qmp_locked, or busy -- however you want to spell it), and
- We want to make sure this bitmap doesn't have a successor.
Now, you and I happen to know that these two conditions aren't actually
independent, but that's not necessarily obvious from this one function
to a new reader. Adding the second check gives some assurance to the reader.
In my mind, the concept of having a successor is now distinct from that
of being busy, and create_successor actually wants to check both things:
(1) That is able to create a successor, because it doesn't have one, and
(2) That it is not modifying a bitmap in use by some operation.
But, you're right, there's no way to have a bitmap with a successor that
isn't busy, so an assertion will suffice for the instructional purpose.
I'll change it.
--js
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 00/10] dirty-bitmaps: deprecate @status field
The current internal meanings of "locked", "user_locked", "qmp_locked", "frozen", "enabled", and "disabled" are all a little muddled. Deprecate the @status field in favor of two new booleans that carry very specific meanings. Then, rename and rework some of the internal semantics to help make the API a bit more clear and easier to read. Well, in my opinion. Based on my current bitmaps branch. V3: [] : patches are identical [] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively 001/10:[0002] [FC] 'block/dirty-bitmap: add recording and busy properties' 002/10:[0002] [FC] 'block/dirty-bitmaps: rename frozen predicate helper' 003/10:[down] 'block/dirty-bitmap: remove set/reset assertions against enabled bit' 004/10:[0006] [FC] 'block/dirty-bitmap: change semantics of enabled predicate' 005/10:[down] 'nbd: change error checking order for bitmaps' 006/10:[0002] [FC] 'block/dirty-bitmap: explicitly lock bitmaps with successors' 007/10:[0011] [FC] 'block/dirty-bitmaps: unify qmp_locked and user_locked calls' 008/10:[0002] [FC] 'block/dirty-bitmaps: move comment block' 009/10:[down] 'blockdev: remove unused paio parameter documentation' 010/10:[down] 'iotests: add busy/recording bit test to 124' V3: (Following Vladimir's feedback) 001: Changed texi phrasing, parameter --> field 002: Commit message adjustment comment edit: "not locked" --> "not user_locked" 003: pulled forward out of patch 004 004: Commit message rewrite create_successor and abdicate doc adjustments 005: New. 006: Change successor doc comment Commit message adjustment 007: BdrvDirtyBitmap struct comment adjustments Comment change to create_successor (lock --> busy) Incidental changes from 004's changes 008: Grammar fix (Eric) 009: New, trivial, unrelated fix tagging along. 010: iotest 124 added. John Snow (10): block/dirty-bitmap: add recording and busy properties block/dirty-bitmaps: rename frozen predicate helper block/dirty-bitmap: remove set/reset assertions against enabled bit block/dirty-bitmap: change semantics of enabled predicate nbd: change error checking order for bitmaps block/dirty-bitmap: explicitly lock bitmaps with successors block/dirty-bitmaps: unify qmp_locked and user_locked calls block/dirty-bitmaps: move comment block blockdev: remove unused paio parameter documentation iotests: add busy/recording bit test to 124 block/dirty-bitmap.c | 111 ++--- blockdev.c | 19 +++--- include/block/dirty-bitmap.h | 7 +-- migration/block-dirty-bitmap.c | 8 +-- nbd/server.c | 14 ++--- qapi/block-core.json | 10 ++- qemu-deprecated.texi | 6 ++ tests/qemu-iotests/124 | 110 tests/qemu-iotests/124.out | 4 +- tests/qemu-iotests/236.out | 28 + 10 files changed, 239 insertions(+), 78 deletions(-) -- 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 03/10] block/dirty-bitmap: remove set/reset assertions against enabled bit
bdrv_set_dirty_bitmap and bdrv_reset_dirty_bitmap are only used as an
internal API by the mirror and migration areas of our code. These
calls modify the bitmap, but do so at the behest of QEMU and not the
guest.
Presently, these bitmaps are always "enabled" anyway, but there's no
reason they have to be.
Modify these internal APIs to drop this assertion.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index aa3f86bb73..9ea5738420 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -542,7 +542,6 @@ int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter)
void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
assert(!bdrv_dirty_bitmap_readonly(bitmap));
hbitmap_set(bitmap->bitmap, offset, bytes);
}
@@ -559,7 +558,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
assert(!bdrv_dirty_bitmap_readonly(bitmap));
hbitmap_reset(bitmap->bitmap, offset, bytes);
}
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 09/10] blockdev: remove unused paio parameter documentation
This field isn't present anymore. Signed-off-by: John Snow --- blockdev.c | 1 - 1 file changed, 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 1aaadb6128..cbce44877d 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1255,7 +1255,6 @@ out_aio_context: * @node: The name of the BDS node to search for bitmaps * @name: The name of the bitmap to search for * @pbs: Output pointer for BDS lookup, if desired. Can be NULL. - * @paio: Output pointer for aio_context acquisition, if desired. Can be NULL. * @errp: Output pointer for error information. Can be NULL. * * @return: A bitmap object on success, or NULL on failure. -- 2.17.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 06/10] block/dirty-bitmap: explicitly lock bitmaps with successors
Instead of implying a user_locked/busy status, make it explicit.
Now, bitmaps in use by migration, NBD or backup operations
are all treated the same way with the same code paths.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
---
block/dirty-bitmap.c | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 976831e765..d92a269753 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -50,7 +50,7 @@ struct BdrvDirtyBitmap {
HBitmap *meta; /* Meta dirty bitmap */
bool qmp_locked;/* Bitmap is locked, it can't be modified
through QMP */
-BdrvDirtyBitmap *successor; /* Anonymous child; implies user_locked state
*/
+BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap, in bytes */
bool disabled; /* Bitmap is disabled. It ignores all writes to
@@ -188,10 +188,8 @@ bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap
*bitmap)
return bitmap->successor;
}
-/* Both conditions disallow user-modification via QMP. */
bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_has_successor(bitmap) ||
- bdrv_dirty_bitmap_qmp_locked(bitmap);
+return bdrv_dirty_bitmap_qmp_locked(bitmap);
}
void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked)
@@ -267,8 +265,9 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
child->disabled = bitmap->disabled;
bitmap->disabled = true;
-/* Install the successor and freeze the parent */
+/* Install the successor and lock the parent */
bitmap->successor = child;
+bitmap->qmp_locked = true;
return 0;
}
@@ -322,6 +321,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
bitmap->successor = NULL;
successor->persistent = bitmap->persistent;
bitmap->persistent = false;
+bitmap->qmp_locked = false;
bdrv_release_dirty_bitmap(bs, bitmap);
return successor;
@@ -351,6 +351,7 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
}
parent->disabled = successor->disabled;
+parent->qmp_locked = false;
bdrv_release_dirty_bitmap_locked(successor);
parent->successor = NULL;
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 10/10] iotests: add busy/recording bit test to 124
This adds a simple test that ensures the busy bit works for push backups,
as well as doubling as bonus test for incremental backups that get interrupted
by EIO errors.
Recording bit tests are already handled sufficiently by 236.
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 110 +
tests/qemu-iotests/124.out | 4 +-
2 files changed, 112 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 5aa1bf1bd6..30f12a2202 100755
--- a/tests/qemu-iotests/124
+++ b/tests/qemu-iotests/124
@@ -634,6 +634,116 @@ class
TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
self.vm.shutdown()
self.check_backups()
+def test_incremental_pause(self):
+"""
+Test an incremental backup that errors into a pause and is resumed.
+"""
+
+drive0 = self.drives[0]
+result = self.vm.qmp('blockdev-add',
+ node_name=drive0['id'],
+ driver=drive0['fmt'],
+ file={
+ 'driver': 'blkdebug',
+ 'image': {
+ 'driver': 'file',
+ 'filename': drive0['file']
+ },
+ 'set-state': [{
+ 'event': 'flush_to_disk',
+ 'state': 1,
+ 'new_state': 2
+ },{
+ 'event': 'read_aio',
+ 'state': 2,
+ 'new_state': 3
+ }],
+ 'inject-error': [{
+ 'event': 'read_aio',
+ 'errno': 5,
+ 'state': 3,
+ 'immediately': False,
+ 'once': True
+ }],
+ })
+self.assert_qmp(result, 'return', {})
+self.create_anchor_backup(drive0)
+bitmap = self.add_bitmap('bitmap0', drive0)
+
+# Emulate guest activity
+self.hmp_io_writes(drive0['id'], (('0xab', 0, 512),
+ ('0xfe', '16M', '256k'),
+ ('0x64', '32736k', '64k')))
+
+# For the purposes of query-block visibility of bitmaps, add a drive
+# frontend after we've written data; otherwise we can't use hmp-io
+result = self.vm.qmp("device_add",
+ id="device0",
+ drive=drive0['id'],
+ driver="virtio-blk")
+self.assert_qmp(result, 'return', {})
+
+# Bitmap Status Check
+query = self.vm.qmp('query-block')
+ret = [bmap for bmap in query['return'][0]['dirty-bitmaps']
+ if bmap.get('name') == bitmap.name][0]
+self.assert_qmp(ret, 'count', 458752)
+self.assert_qmp(ret, 'granularity', 65536)
+self.assert_qmp(ret, 'status', 'active')
+self.assert_qmp(ret, 'busy', False)
+self.assert_qmp(ret, 'recording', True)
+
+# Start backup
+parent, _ = bitmap.last_target()
+target = self.prepare_backup(bitmap, parent)
+res = self.vm.qmp('drive-backup',
+ job_id=bitmap.drive['id'],
+ device=bitmap.drive['id'],
+ sync='incremental',
+ bitmap=bitmap.name,
+ format=bitmap.drive['fmt'],
+ target=target,
+ mode='existing',
+ on_source_error='stop')
+self.assert_qmp(res, 'return', {})
+
+# Wait for the error
+event = self.vm.event_wait(name="BLOCK_JOB_ERROR",
+
match={"data":{"device":bitmap.drive['id']}})
+self.assert_qmp(event, 'data', {'device': bitmap.drive['id'],
+
[libvirt] [PATCH v3 07/10] block/dirty-bitmaps: unify qmp_locked and user_locked calls
These mean the same thing now. Unify them and rename the merged call
bdrv_dirty_bitmap_busy to indicate semantically what we are describing,
as well as help disambiguate from the various _locked and _unlocked
versions of bitmap helpers that refer to mutex locks.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 40 +++---
blockdev.c | 18 +++
include/block/dirty-bitmap.h | 5 ++---
migration/block-dirty-bitmap.c | 6 ++---
nbd/server.c | 6 ++---
5 files changed, 34 insertions(+), 41 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index d92a269753..b3627b0d8c 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -48,8 +48,7 @@ struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
HBitmap *meta; /* Meta dirty bitmap */
-bool qmp_locked;/* Bitmap is locked, it can't be modified
- through QMP */
+bool busy; /* Bitmap is busy, it can't be used via QMP */
BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap, in bytes */
@@ -188,22 +187,17 @@ bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap
*bitmap)
return bitmap->successor;
}
-bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_qmp_locked(bitmap);
+bool bdrv_dirty_bitmap_busy(BdrvDirtyBitmap *bitmap) {
+return bitmap->busy;
}
-void bdrv_dirty_bitmap_set_qmp_locked(BdrvDirtyBitmap *bitmap, bool qmp_locked)
+void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy)
{
qemu_mutex_lock(bitmap->mutex);
-bitmap->qmp_locked = qmp_locked;
+bitmap->busy = busy;
qemu_mutex_unlock(bitmap->mutex);
}
-bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap *bitmap)
-{
-return bitmap->qmp_locked;
-}
-
/* Called with BQL taken. */
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
{
@@ -215,7 +209,7 @@ DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap
*bitmap)
{
if (bdrv_dirty_bitmap_has_successor(bitmap)) {
return DIRTY_BITMAP_STATUS_FROZEN;
-} else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) {
+} else if (bdrv_dirty_bitmap_busy(bitmap)) {
return DIRTY_BITMAP_STATUS_LOCKED;
} else if (!bdrv_dirty_bitmap_enabled(bitmap)) {
return DIRTY_BITMAP_STATUS_DISABLED;
@@ -243,7 +237,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
uint64_t granularity;
BdrvDirtyBitmap *child;
-if (bdrv_dirty_bitmap_user_locked(bitmap)) {
+if (bdrv_dirty_bitmap_busy(bitmap)) {
error_setg(errp, "Cannot create a successor for a bitmap that is
in-use "
"by an operation");
return -1;
@@ -265,9 +259,9 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
child->disabled = bitmap->disabled;
bitmap->disabled = true;
-/* Install the successor and lock the parent */
+/* Install the successor and mark the parent as busy */
bitmap->successor = child;
-bitmap->qmp_locked = true;
+bitmap->busy = true;
return 0;
}
@@ -289,7 +283,7 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap
*bitmap)
static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
assert(!bitmap->active_iterators);
-assert(!bdrv_dirty_bitmap_user_locked(bitmap));
+assert(!bdrv_dirty_bitmap_busy(bitmap));
assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
@@ -321,7 +315,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
bitmap->successor = NULL;
successor->persistent = bitmap->persistent;
bitmap->persistent = false;
-bitmap->qmp_locked = false;
+bitmap->busy = false;
bdrv_release_dirty_bitmap(bs, bitmap);
return successor;
@@ -330,7 +324,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
/**
* In cases of failure where we can no longer safely delete the parent,
* we may wish to re-join the parent and child/successor.
- * The merged parent will not be user_locked.
+ * The merged parent will be marked as not busy.
* The marged parent will be enabled if and only if the successor was enabled.
* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken.
*/
@@ -351,7 +345,7 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
}
parent->disabled = successor->disabled;
-parent->qmp_locked = false;
+parent->busy = false;
bdrv_release_dirty_bitmap_locked(successor);
parent->successor = NULL;
@@ -382,7 +376,7 @@ void bdrv_dirty_bitmap_tr
[libvirt] [PATCH v3 08/10] block/dirty-bitmaps: move comment block
Simply move the big status enum comment block to above the status
function, and document it as being deprecated. The whole confusing
block can get deleted in three releases time.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 36 +++-
1 file changed, 19 insertions(+), 17 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index b3627b0d8c..86c3b87ab9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -28,22 +28,6 @@
#include "block/block_int.h"
#include "block/blockjob.h"
-/**
- * A BdrvDirtyBitmap can be in four possible user-visible states:
- * (1) Active: successor is NULL, and disabled is false: full r/w mode
- * (2) Disabled: successor is NULL, and disabled is true: qualified r/w mode,
- * guest writes are dropped, but monitor writes are possible,
- * through commands like merge and clear.
- * (3) Frozen: successor is not NULL.
- * A frozen bitmap cannot be renamed, deleted, cleared, set,
- * enabled, merged to, etc. A frozen bitmap can only abdicate()
- * or reclaim().
- * In this state, the anonymous successor bitmap may be either
- * Active and recording writes from the guest (e.g. backup jobs),
- * but it can be Disabled and not recording writes.
- * (4) Locked: Whether Active or Disabled, the user cannot modify this bitmap
- * in any way from the monitor.
- */
struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
@@ -204,7 +188,25 @@ bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
return !bitmap->disabled;
}
-/* Called with BQL taken. */
+/**
+ * bdrv_dirty_bitmap_status: This API is now deprecated.
+ * Called with BQL taken.
+ *
+ * A BdrvDirtyBitmap can be in four possible user-visible states:
+ * (1) Active: successor is NULL, and disabled is false: full r/w mode
+ * (2) Disabled: successor is NULL, and disabled is true: qualified r/w mode,
+ * guest writes are dropped, but monitor writes are possible,
+ * through commands like merge and clear.
+ * (3) Frozen: successor is not NULL.
+ * A frozen bitmap cannot be renamed, deleted, cleared, set,
+ * enabled, merged to, etc. A frozen bitmap can only abdicate()
+ * or reclaim().
+ * In this state, the anonymous successor bitmap may be either
+ * Active and recording writes from the guest (e.g. backup jobs),
+ * or it can be Disabled and not recording writes.
+ * (4) Locked: Whether Active or Disabled, the user cannot modify this bitmap
+ * in any way from the monitor.
+ */
DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap)
{
if (bdrv_dirty_bitmap_has_successor(bitmap)) {
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 02/10] block/dirty-bitmaps: rename frozen predicate helper
"Frozen" was a good description a long time ago, but it isn't adequate now.
Rename the frozen predicate to has_successor to make the semantics of the
predicate more clear to outside callers.
In the process, remove some calls to frozen() that no longer semantically
make sense. For bdrv_enable_dirty_bitmap_locked and
bdrv_disable_dirty_bitmap_locked, it doesn't make sense to prohibit QEMU
internals from performing this action when we only wished to prohibit QMP
users from issuing these commands. All of the QMP API commands for bitmap
manipulation already check against user_locked() to prohibit these actions.
Several other assertions really want to check that the bitmap isn't in-use
by another operation -- use the bitmap_user_locked function for this instead,
which presently also checks for has_successor. This leaves some redundant
checks of has_sucessor through different helpers that are addressed in
forthcoming patches.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 32 +---
include/block/dirty-bitmap.h | 2 +-
migration/block-dirty-bitmap.c | 2 +-
3 files changed, 19 insertions(+), 17 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 101383b3af..aa3f86bb73 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -50,7 +50,7 @@ struct BdrvDirtyBitmap {
HBitmap *meta; /* Meta dirty bitmap */
bool qmp_locked;/* Bitmap is locked, it can't be modified
through QMP */
-BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
+BdrvDirtyBitmap *successor; /* Anonymous child; implies user_locked state
*/
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap, in bytes */
bool disabled; /* Bitmap is disabled. It ignores all writes to
@@ -183,14 +183,14 @@ const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap
*bitmap)
}
/* Called with BQL taken. */
-bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap)
+bool bdrv_dirty_bitmap_has_successor(BdrvDirtyBitmap *bitmap)
{
return bitmap->successor;
}
/* Both conditions disallow user-modification via QMP. */
bool bdrv_dirty_bitmap_user_locked(BdrvDirtyBitmap *bitmap) {
-return bdrv_dirty_bitmap_frozen(bitmap) ||
+return bdrv_dirty_bitmap_has_successor(bitmap) ||
bdrv_dirty_bitmap_qmp_locked(bitmap);
}
@@ -215,7 +215,7 @@ bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
/* Called with BQL taken. */
DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap)
{
-if (bdrv_dirty_bitmap_frozen(bitmap)) {
+if (bdrv_dirty_bitmap_has_successor(bitmap)) {
return DIRTY_BITMAP_STATUS_FROZEN;
} else if (bdrv_dirty_bitmap_qmp_locked(bitmap)) {
return DIRTY_BITMAP_STATUS_LOCKED;
@@ -235,7 +235,7 @@ static bool bdrv_dirty_bitmap_recording(BdrvDirtyBitmap
*bitmap)
/**
* Create a successor bitmap destined to replace this bitmap after an
operation.
- * Requires that the bitmap is not frozen and has no successor.
+ * Requires that the bitmap is not user_locked and has no successor.
* Called with BQL taken.
*/
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
@@ -244,12 +244,16 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState
*bs,
uint64_t granularity;
BdrvDirtyBitmap *child;
-if (bdrv_dirty_bitmap_frozen(bitmap)) {
-error_setg(errp, "Cannot create a successor for a bitmap that is "
- "currently frozen");
+if (bdrv_dirty_bitmap_user_locked(bitmap)) {
+error_setg(errp, "Cannot create a successor for a bitmap that is
in-use "
+ "by an operation");
+return -1;
+}
+if (bdrv_dirty_bitmap_has_successor(bitmap)) {
+error_setg(errp, "Cannot create a successor for a bitmap that already "
+ "has one");
return -1;
}
-assert(!bitmap->successor);
/* Create an anonymous successor */
granularity = bdrv_dirty_bitmap_granularity(bitmap);
@@ -268,7 +272,6 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
-assert(!bdrv_dirty_bitmap_frozen(bitmap));
bitmap->disabled = false;
}
@@ -285,7 +288,7 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap
*bitmap)
static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
{
assert(!bitmap->active_iterators);
-assert(!bdrv_dirty_bitmap_frozen(bitmap));
+assert(!bdrv_dirty_bitmap_user_locked(bitmap));
assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
@@ -325,7 +328,7 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
/**
* In cases of failur
[libvirt] [PATCH v3 04/10] block/dirty-bitmap: change semantics of enabled predicate
Currently, the enabled predicate means something like:
"the QAPI status of the bitmap is ACTIVE."
After this patch, it should mean exclusively:
"This bitmap is recording guest writes, and is allowed to do so."
In many places, this is how this predicate was already used.
Internal usages of the bitmap QPI can call user_locked to find out if
the bitmap is in use by an operation.
To accommodate this, modify the create_successor routine to now
explicitly disable the parent bitmap at creation time.
Justifications:
1. bdrv_dirty_bitmap_status suffers no change from the lack of
1:1 parity with the new predicates because of the order in which
the predicates are checked. This is now only for compatibility.
2. bdrv_set_dirty() is unchanged: pre-patch, it was skipping bitmaps that were
disabled or had a successor, while post-patch it is only skipping bitmaps
that are disabled. To accommodate this, create_successor now ensures that
any bitmap with a successor is explicitly disabled.
3. qcow2_store_persistent_dirty_bitmaps: No functional change. This function
cares only about the literal enabled bit, and makes no effort to check if
the bitmap is in-use or not. After this patch there are still no ways to
produce an enabled bitmap with a successor.
4. block_dirty_bitmap_enable_prepare
block_dirty_bitmap_disable_prepare
init_dirty_bitmap_migration
nbd_export_new
These functions care about the literal enabled bit,
and already check user_locked separately.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 9 +++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 9ea5738420..976831e765 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -209,7 +209,7 @@ bool bdrv_dirty_bitmap_qmp_locked(BdrvDirtyBitmap *bitmap)
/* Called with BQL taken. */
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
{
-return !(bitmap->disabled || bitmap->successor);
+return !bitmap->disabled;
}
/* Called with BQL taken. */
@@ -236,6 +236,7 @@ static bool bdrv_dirty_bitmap_recording(BdrvDirtyBitmap
*bitmap)
/**
* Create a successor bitmap destined to replace this bitmap after an
operation.
* Requires that the bitmap is not user_locked and has no successor.
+ * The successor will be enabled if the parent bitmap was.
* Called with BQL taken.
*/
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
@@ -264,6 +265,7 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
/* Successor will be on or off based on our current state. */
child->disabled = bitmap->disabled;
+bitmap->disabled = true;
/* Install the successor and freeze the parent */
bitmap->successor = child;
@@ -328,7 +330,8 @@ BdrvDirtyBitmap
*bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
/**
* In cases of failure where we can no longer safely delete the parent,
* we may wish to re-join the parent and child/successor.
- * The merged parent will not be user_locked, nor explicitly re-enabled.
+ * The merged parent will not be user_locked.
+ * The marged parent will be enabled if and only if the successor was enabled.
* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken.
*/
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
@@ -346,6 +349,8 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
error_setg(errp, "Merging of parent and successor bitmap failed");
return NULL;
}
+
+parent->disabled = successor->disabled;
bdrv_release_dirty_bitmap_locked(successor);
parent->successor = NULL;
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3 01/10] block/dirty-bitmap: add recording and busy properties
The current API allows us to report a single status, which we've defined as:
Frozen: has a successor, treated as qmp_locked, may or may not be enabled.
Locked: no successor, qmp_locked. may or may not be enabled.
Disabled: Not frozen or locked, disabled.
Active: Not frozen, locked, or disabled.
The problem is that both "Frozen" and "Locked" mean nearly the same thing,
and that both of them do not intuit whether they are recording guest writes
or not.
This patch deprecates that status field and introduces two orthogonal
properties instead to replace it.
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 9 +
qapi/block-core.json | 10 +-
qemu-deprecated.texi | 6 ++
tests/qemu-iotests/236.out | 28
4 files changed, 52 insertions(+), 1 deletion(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index c6d4acebfa..101383b3af 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -226,6 +226,13 @@ DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap
*bitmap)
}
}
+/* Called with BQL taken. */
+static bool bdrv_dirty_bitmap_recording(BdrvDirtyBitmap *bitmap)
+{
+return !bitmap->disabled || (bitmap->successor &&
+ !bitmap->successor->disabled);
+}
+
/**
* Create a successor bitmap destined to replace this bitmap after an
operation.
* Requires that the bitmap is not frozen and has no successor.
@@ -448,6 +455,8 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
info->has_name = !!bm->name;
info->name = g_strdup(bm->name);
info->status = bdrv_dirty_bitmap_status(bm);
+info->recording = bdrv_dirty_bitmap_recording(bm);
+info->busy = bdrv_dirty_bitmap_user_locked(bm);
info->persistent = bm->persistent;
entry->value = info;
*plist = entry;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 2b8afbb924..6e543594b3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -458,7 +458,14 @@
#
# @granularity: granularity of the dirty bitmap in bytes (since 1.4)
#
-# @status: current status of the dirty bitmap (since 2.4)
+# @status: Deprecated in favor of @recording and @locked. (since 2.4)
+#
+# @recording: true if the bitmap is recording new writes from the guest.
+# Replaces `active` and `disabled` statuses. (since 4.0)
+#
+# @busy: true if the bitmap is in-use by some operation (NBD or jobs)
+#and cannot be modified via QMP or used by another operation.
+#Replaces `locked` and `frozen` statuses. (since 4.0)
#
# @persistent: true if the bitmap will eventually be flushed to persistent
# storage (since 4.0)
@@ -467,6 +474,7 @@
##
{ 'struct': 'BlockDirtyInfo',
'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
+ 'recording': 'bool', 'busy': 'bool',
'status': 'DirtyBitmapStatus', 'persistent': 'bool' } }
##
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 45c57952da..4c7ae8180f 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -67,6 +67,12 @@ topologies described with -smp include all possible cpus,
i.e.
"autoload" parameter is now ignored. All bitmaps are automatically loaded
from qcow2 images.
+@subsection query-block result field(s) dirty-bitmaps[i].status (since 4.0)
+
+The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
+the query-block command is deprecated. Two new boolean fields,
+``recording'' and ``busy'' effectively replace it.
+
@subsection query-cpus (since 2.12.0)
The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command.
diff --git a/tests/qemu-iotests/236.out b/tests/qemu-iotests/236.out
index 5006f7bca1..815cd053f0 100644
--- a/tests/qemu-iotests/236.out
+++ b/tests/qemu-iotests/236.out
@@ -22,17 +22,21 @@ write -P0xcd 0x3ff 64k
"bitmaps": {
"drive0": [
{
+"busy": false,
"count": 262144,
"granularity": 65536,
"name": "bitmapB",
"persistent": false,
+"recording": true,
"status": "active"
},
{
+"busy": false,
"count": 262144,
"granularity": 65536,
"name": "bitmapA",
"persistent": false,
+"recording": true,
"status": "active"
}
]
@@ -84,17 +88,21 @@ write -P0xcd 0x3ff 64k
"bitmaps": {
[libvirt] [PATCH v3 05/10] nbd: change error checking order for bitmaps
Check that the bitmap is not in use prior to it checking if it is
not enabled/recording guest writes. The bitmap being busy was likely
at the behest of the user, so this error has a greater chance of being
understood by the user.
Signed-off-by: John Snow
---
nbd/server.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/nbd/server.c b/nbd/server.c
index 0910d09a6d..de21c64ce3 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -1510,6 +1510,11 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t
dev_offset,
goto fail;
}
+if (bdrv_dirty_bitmap_user_locked(bm)) {
+error_setg(errp, "Bitmap '%s' is in use", bitmap);
+goto fail;
+}
+
if ((nbdflags & NBD_FLAG_READ_ONLY) && bdrv_is_writable(bs) &&
bdrv_dirty_bitmap_enabled(bm)) {
error_setg(errp,
@@ -1518,11 +1523,6 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t
dev_offset,
goto fail;
}
-if (bdrv_dirty_bitmap_user_locked(bm)) {
-error_setg(errp, "Bitmap '%s' is in use", bitmap);
-goto fail;
-}
-
bdrv_dirty_bitmap_set_qmp_locked(bm, true);
exp->export_bitmap = bm;
exp->export_bitmap_context = g_strdup_printf("qemu:dirty-bitmap:%s",
--
2.17.2
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 02/10] block/dirty-bitmaps: rename frozen predicate helper
On 2/25/19 2:01 AM, Vladimir Sementsov-Ogievskiy wrote:
> 23.02.2019 3:06, John Snow wrote:
>> "Frozen" was a good description a long time ago, but it isn't adequate now.
>> Rename the frozen predicate to has_successor to make the semantics of the
>> predicate more clear to outside callers.
>>
>> In the process, remove some calls to frozen() that no longer semantically
>> make sense. For bdrv_enable_dirty_bitmap_locked and
>> bdrv_disable_dirty_bitmap_locked, it doesn't make sense to prohibit QEMU
>> internals from performing this action when we only wished to prohibit QMP
>> users from issuing these commands. All of the QMP API commands for bitmap
>> manipulation already check against user_locked() to prohibit these actions.
>>
>> Several other assertions really want to check that the bitmap isn't in-use
>> by another operation -- use the bitmap_user_locked function for this instead,
>> which presently also checks for has_successor. This leaves some redundant
>> checks of has_sucessor through different helpers that are addressed in
>> forthcoming patches.
>>
>> Signed-off-by: John Snow
>> ---
>> block/dirty-bitmap.c | 32 +---
>> include/block/dirty-bitmap.h | 2 +-
>> migration/block-dirty-bitmap.c | 2 +-
>> 3 files changed, 19 insertions(+), 17 deletions(-)
>>
>> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
>> index 101383b3af..aa3f86bb73 100644
>> --- a/block/dirty-bitmap.c
>> +++ b/block/dirty-bitmap.c
>
> [..]
>
>> @@ -285,7 +288,7 @@ void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap
>> *bitmap)
>> static void bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
>> {
>> assert(!bitmap->active_iterators);
>> -assert(!bdrv_dirty_bitmap_frozen(bitmap));
>> +assert(!bdrv_dirty_bitmap_user_locked(bitmap));
>
> hmm. I'd prefere to keep a separate assertion for successor: we don't free it
> here,
> so we really depend on absence of this object.
>
Reasonable.
>> assert(!bitmap->meta);
>> QLIST_REMOVE(bitmap, list);
>> hbitmap_free(bitmap->bitmap);
>> @@ -325,7 +328,7 @@ BdrvDirtyBitmap
>> *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
>> /**
>>* In cases of failure where we can no longer safely delete the parent,
>>* we may wish to re-join the parent and child/successor.
>> - * The merged parent will be un-frozen, but not explicitly re-enabled.
>> + * The merged parent will not be user_locked, nor explicitly re-enabled.
>>* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken.
>>*/
>> BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
>> @@ -373,7 +376,7 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs,
>> int64_t bytes)
>>
>> bdrv_dirty_bitmaps_lock(bs);
>> QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
>> -assert(!bdrv_dirty_bitmap_frozen(bitmap));
>> +assert(!bdrv_dirty_bitmap_user_locked(bitmap));
>
> and here too. As we don't truncate successors.
>
>> assert(!bitmap->active_iterators);
>> hbitmap_truncate(bitmap->bitmap, bytes);
>> bitmap->size = bytes;
>> @@ -391,7 +394,7 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs,
>> BdrvDirtyBitmap *bitmap)
>>
>> /**
>>* Release all named dirty bitmaps attached to a BDS (for use in
>> bdrv_close()).
>> - * There must not be any frozen bitmaps attached.
>> + * There must not be any user_locked bitmaps attached.
>>* This function does not remove persistent bitmaps from the storage.
>>* Called with BQL taken.
>>*/
>> @@ -428,7 +431,6 @@ void
>> bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs,
>> void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
>> {
>> bdrv_dirty_bitmap_lock(bitmap);
>> -assert(!bdrv_dirty_bitmap_frozen(bitmap));
>> bitmap->disabled = true;
>> bdrv_dirty_bitmap_unlock(bitmap);
>> }
>> diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
>> index 04a117fc81..cdbb4dfefd 100644
>> --- a/include/block/dirty-bitmap.h
>> +++ b/include/block/dirty-bitmap.h
>
> Could you please add scripts/git.orderfile to your .git/config, like
>
> [diff]
> orderFile = /path/to/scripts/git.orderfile
>
> ?
>
Hm, we should add this to the .gitpublish profile, but yes, let me
re-add this.
>> @@ -36,7 +36,
Re: [libvirt] [PATCH v3 10/10] iotests: add busy/recording bit test to 124
On 2/23/19 5:06 PM, Eric Blake wrote:
> On 2/22/19 6:06 PM, John Snow wrote:
>> This adds a simple test that ensures the busy bit works for push backups,
>> as well as doubling as bonus test for incremental backups that get
>> interrupted
>> by EIO errors.
>
> This makes 124 longer to run, but it is not in the 'quick' group, so
> that's okay.
>
> The easy part: I compiled the series, and validated that ./check still
> passes with this applied, so:
>
> Tested-by: Eric Blake
>
> Now for the fun part (I know from IRC that you had some interesting
> challenges coming up with scenarios to even provoke the states you
> wanted shown in the output):
>
>>
>> Recording bit tests are already handled sufficiently by 236.
>>
>> Signed-off-by: John Snow
>> ---
>> tests/qemu-iotests/124 | 110 +
>> tests/qemu-iotests/124.out | 4 +-
>> 2 files changed, 112 insertions(+), 2 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
>> index 5aa1bf1bd6..30f12a2202 100755
>> --- a/tests/qemu-iotests/124
>> +++ b/tests/qemu-iotests/124
>> @@ -634,6 +634,116 @@ class
>> TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
>> self.vm.shutdown()
>> self.check_backups()
>>
>> +def test_incremental_pause(self):
>> +"""
>> +Test an incremental backup that errors into a pause and is resumed.
>> +"""
>> +
>> +drive0 = self.drives[0]
>> +result = self.vm.qmp('blockdev-add',
>> + node_name=drive0['id'],
>> + driver=drive0['fmt'],
>> + file={
>> + 'driver': 'blkdebug',
>> + 'image': {
>> + 'driver': 'file',
>> + 'filename': drive0['file']
>> + },
>> + 'set-state': [{
>> + 'event': 'flush_to_disk',
>> + 'state': 1,
>> + 'new_state': 2
>> + },{
>> + 'event': 'read_aio',
>> + 'state': 2,
>> + 'new_state': 3
>> + }],
>> + 'inject-error': [{
>> + 'event': 'read_aio',
>> + 'errno': 5,
>> + 'state': 3,
>> + 'immediately': False,
>> + 'once': True
>> + }],
>
> Yeah, it's hairy to come up with sequences that will pause at the right
> place, and this may be sensitive enough that we have to revisit it in
> the future, but for now it works and I don't have any better suggestions.
>
The problem in this case was that the drive_add itself provokes a
read_aio. I didn't dig too far to see why, but I needed an extra
read_aio state in the sequence there so that the error would occur
during the actual backup job.
I think I'll document this in the test, with at least a comment.
I wonder if there's a syntactical sugar we could write that makes this
easier to read, like:
'inject-error': [{ 'event': 'flush_to_disk->read_aio->read_aio' }] or
some such, but that's probably at odds with the existing interface and
not worth my time to go on a crusade about.
>> + })
>> +self.assert_qmp(result, 'return', {})
>> +self.create_anchor_backup(drive0)
>> +bitmap = self.add_bitmap('bitmap0', drive0)
>> +
>> +# Emulate guest activity
>> +self.hmp_io_writes(drive0['id'], (('0xab', 0, 512),
>> + ('0xfe', '16M', '256k'),
>> + ('0x64', '32736k', '64k')))
>> +
>> +# For the purposes of
Re: [libvirt] [PATCH v3 07/10] block/dirty-bitmaps: unify qmp_locked and user_locked calls
On 2/25/19 7:03 AM, Vladimir Sementsov-Ogievskiy wrote: > 23.02.2019 3:06, John Snow wrote: >> These mean the same thing now. Unify them and rename the merged call >> bdrv_dirty_bitmap_busy to indicate semantically what we are describing, >> as well as help disambiguate from the various _locked and _unlocked >> versions of bitmap helpers that refer to mutex locks. >> >> Signed-off-by: John Snow > > Hmm, and here, you directly fixed what I've noticed, so my r-b, > of course, still applies. > Sorry for not adding them. I didn't plan to merge the series until you got a chance to review it, so I wasn't worried about missing it in the long run. Sorry if it makes it harder to see which ones you'd like to look at. > Ha, but I noticed funny thing: > >> --- >> block/dirty-bitmap.c | 40 +++--- >> blockdev.c | 18 +++ >> include/block/dirty-bitmap.h | 5 ++--- >> migration/block-dirty-bitmap.c | 6 ++--- >> nbd/server.c | 6 ++--- >> 5 files changed, 34 insertions(+), 41 deletions(-) >> >> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c >> index d92a269753..b3627b0d8c 100644 >> --- a/block/dirty-bitmap.c >> +++ b/block/dirty-bitmap.c > > [..] > >> @@ -265,9 +259,9 @@ int bdrv_dirty_bitmap_create_successor(BlockDriverState >> *bs, >> child->disabled = bitmap->disabled; >> bitmap->disabled = true; >> >> -/* Install the successor and lock the parent */ >> +/* Install the successor and mark the parent as busy */ > > I asked you to fix comment for bdrv_dirty_bitmap_create_successor(), but I > didn't > noticed this one, I meant the comment to the whole function, placed above it, > which > now have > "Requires that the bitmap is not user_locked and has no successor." > > So, it should be updated too. > > and with it: > Reviewed-by: Vladimir Sementsov-Ogievskiy > Ha. OK, updated to "is not marked busy" Thanks for the reviews! >> bitmap->successor = child; >> -bitmap->qmp_locked = true; >> +bitmap->busy = true; >> return 0; >> } >> > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3 00/10] dirty-bitmaps: deprecate @status field
On 2/22/19 7:06 PM, John Snow wrote: > The current internal meanings of "locked", "user_locked", > "qmp_locked", "frozen", "enabled", and "disabled" are all > a little muddled. > > Deprecate the @status field in favor of two new booleans > that carry very specific meanings. Then, rename and rework > some of the internal semantics to help make the API a bit > more clear and easier to read. > > Well, in my opinion. > > Based on my current bitmaps branch. > > V3: > > [] : patches are identical > [] : number of functional differences between upstream/downstream patch > [down] : patch is downstream-only > The flags [FC] indicate (F)unctional and (C)ontextual differences, > respectively > > 001/10:[0002] [FC] 'block/dirty-bitmap: add recording and busy properties' > 002/10:[0002] [FC] 'block/dirty-bitmaps: rename frozen predicate helper' > 003/10:[down] 'block/dirty-bitmap: remove set/reset assertions against > enabled bit' > 004/10:[0006] [FC] 'block/dirty-bitmap: change semantics of enabled predicate' > 005/10:[down] 'nbd: change error checking order for bitmaps' > 006/10:[0002] [FC] 'block/dirty-bitmap: explicitly lock bitmaps with > successors' > 007/10:[0011] [FC] 'block/dirty-bitmaps: unify qmp_locked and user_locked > calls' > 008/10:[0002] [FC] 'block/dirty-bitmaps: move comment block' > 009/10:[down] 'blockdev: remove unused paio parameter documentation' > 010/10:[down] 'iotests: add busy/recording bit test to 124' > > V3: (Following Vladimir's feedback) > > 001: Changed texi phrasing, parameter --> field > 002: Commit message adjustment > comment edit: "not locked" --> "not user_locked" > 003: pulled forward out of patch 004 > 004: Commit message rewrite > create_successor and abdicate doc adjustments > 005: New. > 006: Change successor doc comment > Commit message adjustment > 007: BdrvDirtyBitmap struct comment adjustments > Comment change to create_successor (lock --> busy) > Incidental changes from 004's changes > 008: Grammar fix (Eric) > 009: New, trivial, unrelated fix tagging along. > 010: iotest 124 added. > > John Snow (10): > block/dirty-bitmap: add recording and busy properties > block/dirty-bitmaps: rename frozen predicate helper > block/dirty-bitmap: remove set/reset assertions against enabled bit > block/dirty-bitmap: change semantics of enabled predicate > nbd: change error checking order for bitmaps > block/dirty-bitmap: explicitly lock bitmaps with successors > block/dirty-bitmaps: unify qmp_locked and user_locked calls > block/dirty-bitmaps: move comment block > blockdev: remove unused paio parameter documentation > iotests: add busy/recording bit test to 124 > > block/dirty-bitmap.c | 111 ++--- > blockdev.c | 19 +++--- > include/block/dirty-bitmap.h | 7 +-- > migration/block-dirty-bitmap.c | 8 +-- > nbd/server.c | 14 ++--- > qapi/block-core.json | 10 ++- > qemu-deprecated.texi | 6 ++ > tests/qemu-iotests/124 | 110 > tests/qemu-iotests/124.out | 4 +- > tests/qemu-iotests/236.out | 28 + > 10 files changed, 239 insertions(+), 78 deletions(-) > Pushed to my bitmaps branch with the following minor changes: 001/10:[0002] [FC] 'block/dirty-bitmap: add recording and busy properties' 002/10:[0002] [FC] 'block/dirty-bitmaps: rename frozen predicate helper' 003/10:[] [--] 'block/dirty-bitmap: remove set/reset assertions against enabled bit' 004/10:[] [--] 'block/dirty-bitmap: change semantics of enabled predicate' 005/10:[] [--] 'nbd: change error checking order for bitmaps' 006/10:[] [--] 'block/dirty-bitmap: explicitly lock bitmaps with successors' 007/10:[0002] [FC] 'block/dirty-bitmaps: unify qmp_locked and user_locked calls' 008/10:[] [--] 'block/dirty-bitmaps: move comment block' 009/10:[] [--] 'blockdev: remove unused paio parameter documentation' 010/10:[0003] [FC] 'iotests: add busy/recording bit test to 124' 001: Texi doc spelling (Eric) 002: Commit message spelling (Eric) Additional asserts (Vladimir) 007: Comment phrasing (Vladimir) 010: Extra test comment -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2] qapi: add dirty-bitmaps to query-named-block-nodes result
From: Vladimir Sementsov-Ogievskiy
Let's add a possibility to query dirty-bitmaps not only on root nodes.
It is useful when dealing both with snapshots and incremental backups.
Signed-off-by: Vladimir Sementsov-Ogievskiy
[Added deprecation and feature flag information. --js]
Signed-off-by: John Snow
---
block/qapi.c | 5 +
qapi/block-core.json | 14 +-
qemu-deprecated.texi | 12
3 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/block/qapi.c b/block/qapi.c
index 917435f022..15f1030264 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
info->backing_file = g_strdup(bs->backing_file);
}
+if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
+info->has_dirty_bitmaps = true;
+info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
+}
+
info->detect_zeroes = bs->detect_zeroes;
if (blk && blk_get_public(blk)->throttle_group_member.throttle_state) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0d43d4f37c..0d67dd245c 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -360,6 +360,16 @@
# @write_threshold: configured write threshold for the device.
# 0 if disabled. (Since 2.3)
#
+# @dirty-bitmaps: dirty bitmaps information (only present if node
+# has one or more dirty bitmaps) (Since 4.2)
+#
+# Features:
+# @node-dirty-bitmaps: Signals the capability to return dirty bitmap
information
+# per-node instead of per-drive. If this flag is present,
+# dirty-bitmaps should not be read from the BlockInfo
+# structure, the top-level data for query-block.
+# (Since 4.2)
+#
# Since: 0.14.0
#
##
@@ -378,7 +388,8 @@
'*bps_wr_max_length': 'int', '*iops_max_length': 'int',
'*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int',
'*iops_size': 'int', '*group': 'str', 'cache': 'BlockdevCacheInfo',
-'write_threshold': 'int' } }
+'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo'] },
+ 'features': [ { 'name': 'node-dirty-bitmaps' } ] }
##
# @BlockDeviceIoStatus:
@@ -656,6 +667,7 @@
#
# @dirty-bitmaps: dirty bitmaps information (only present if the
# driver has one or more dirty bitmaps) (Since 2.0)
+# Deprecated in 4.2; see BlockDirtyInfo instead.
#
# @io-status: @BlockDeviceIoStatus. Only present if the device
# supports it and the VM is configured to stop on errors
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index c90b08d553..bc4e5ac1d7 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo'' structure,
returned by
the query-block command is deprecated. Two new boolean fields,
``recording'' and ``busy'' effectively replace it.
+@subsection query-block result field dirty-bitmaps (Since 4.2)
+
+The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by
+the query-block command is itself now deprecated. The ``dirty-bitmaps``
+field of the ``BlockDeviceInfo`` struct should be used instead, which is the
+type of the ``inserted`` field in query-block replies, as well as the
+type of array items in query-named-block-nodes.
+
+In the absence of bitmaps on either structure, management APIs may use the
+presence of the ``node-dirty-bitmaps`` feature flag on the ``BlockDeviceInfo``
+structure to know where to anticipate bitmap data when present.
+
@subsection query-cpus (since 2.12.0)
The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command.
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
From: Vladimir Sementsov-Ogievskiy
Let's add a possibility to query dirty-bitmaps not only on root nodes.
It is useful when dealing both with snapshots and incremental backups.
Signed-off-by: Vladimir Sementsov-Ogievskiy
[Added deprecation information. --js]
Signed-off-by: John Snow
---
block/qapi.c | 5 +
qapi/block-core.json | 6 +-
qemu-deprecated.texi | 12
3 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/block/qapi.c b/block/qapi.c
index 917435f022..15f1030264 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
info->backing_file = g_strdup(bs->backing_file);
}
+if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
+info->has_dirty_bitmaps = true;
+info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
+}
+
info->detect_zeroes = bs->detect_zeroes;
if (blk && blk_get_public(blk)->throttle_group_member.throttle_state) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0d43d4f37c..9210ae233d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -360,6 +360,9 @@
# @write_threshold: configured write threshold for the device.
# 0 if disabled. (Since 2.3)
#
+# @dirty-bitmaps: dirty bitmaps information (only present if node
+# has one or more dirty bitmaps) (Since 4.2)
+#
# Since: 0.14.0
#
##
@@ -378,7 +381,7 @@
'*bps_wr_max_length': 'int', '*iops_max_length': 'int',
'*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int',
'*iops_size': 'int', '*group': 'str', 'cache': 'BlockdevCacheInfo',
-'write_threshold': 'int' } }
+'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo'] } }
##
# @BlockDeviceIoStatus:
@@ -656,6 +659,7 @@
#
# @dirty-bitmaps: dirty bitmaps information (only present if the
# driver has one or more dirty bitmaps) (Since 2.0)
+# Deprecated in 4.2; see BlockDirtyInfo instead.
#
# @io-status: @BlockDeviceIoStatus. Only present if the device
# supports it and the VM is configured to stop on errors
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index c90b08d553..6374b66546 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo'' structure,
returned by
the query-block command is deprecated. Two new boolean fields,
``recording'' and ``busy'' effectively replace it.
+@subsection query-block result field dirty-bitmaps (Since 4.2)
+
+The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by
+the query-block command is itself now deprecated. The ``dirty-bitmaps``
+field of the ``BlockDeviceInfo`` struct should be used instead, which is the
+type of the ``inserted`` field in query-block replies, as well as the
+type of array items in query-named-block-nodes.
+
+Since the ``dirty-bitmaps`` field is optionally present in both the old and
+new locations, clients must use introspection to learn where to anticipate
+the field if/when it does appear in command output.
+
@subsection query-cpus (since 2.12.0)
The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command.
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/17/19 1:39 PM, John Snow wrote:
> From: Vladimir Sementsov-Ogievskiy
>
> Let's add a possibility to query dirty-bitmaps not only on root nodes.
> It is useful when dealing both with snapshots and incremental backups.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy
> [Added deprecation information. --js]
> Signed-off-by: John Snow
v2: Added feature flag in a bout of excitement over a new feature to
play with. Added deprecation text.
v3: Removed feature flag because this is the real world and not every
new feature is needed for every new patch.
--js
> ---
> block/qapi.c | 5 +
> qapi/block-core.json | 6 +-
> qemu-deprecated.texi | 12
> 3 files changed, 22 insertions(+), 1 deletion(-)
>
> diff --git a/block/qapi.c b/block/qapi.c
> index 917435f022..15f1030264 100644
> --- a/block/qapi.c
> +++ b/block/qapi.c
> @@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
> info->backing_file = g_strdup(bs->backing_file);
> }
>
> +if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
> +info->has_dirty_bitmaps = true;
> +info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
> +}
> +
> info->detect_zeroes = bs->detect_zeroes;
>
> if (blk && blk_get_public(blk)->throttle_group_member.throttle_state) {
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 0d43d4f37c..9210ae233d 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -360,6 +360,9 @@
> # @write_threshold: configured write threshold for the device.
> # 0 if disabled. (Since 2.3)
> #
> +# @dirty-bitmaps: dirty bitmaps information (only present if node
> +# has one or more dirty bitmaps) (Since 4.2)
> +#
> # Since: 0.14.0
> #
> ##
> @@ -378,7 +381,7 @@
> '*bps_wr_max_length': 'int', '*iops_max_length': 'int',
> '*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int',
> '*iops_size': 'int', '*group': 'str', 'cache':
> 'BlockdevCacheInfo',
> -'write_threshold': 'int' } }
> +'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo'] }
> }
>
> ##
> # @BlockDeviceIoStatus:
> @@ -656,6 +659,7 @@
> #
> # @dirty-bitmaps: dirty bitmaps information (only present if the
> # driver has one or more dirty bitmaps) (Since 2.0)
> +# Deprecated in 4.2; see BlockDirtyInfo instead.
> #
> # @io-status: @BlockDeviceIoStatus. Only present if the device
> # supports it and the VM is configured to stop on errors
> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
> index c90b08d553..6374b66546 100644
> --- a/qemu-deprecated.texi
> +++ b/qemu-deprecated.texi
> @@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo''
> structure, returned by
> the query-block command is deprecated. Two new boolean fields,
> ``recording'' and ``busy'' effectively replace it.
>
> +@subsection query-block result field dirty-bitmaps (Since 4.2)
> +
> +The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by
> +the query-block command is itself now deprecated. The ``dirty-bitmaps``
> +field of the ``BlockDeviceInfo`` struct should be used instead, which is the
> +type of the ``inserted`` field in query-block replies, as well as the
> +type of array items in query-named-block-nodes.
> +
> +Since the ``dirty-bitmaps`` field is optionally present in both the old and
> +new locations, clients must use introspection to learn where to anticipate
> +the field if/when it does appear in command output.
> +
> @subsection query-cpus (since 2.12.0)
>
> The ``query-cpus'' command is replaced by the ``query-cpus-fast'' command.
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/17/19 3:13 PM, Eric Blake wrote: > On 7/17/19 12:39 PM, John Snow wrote: >> From: Vladimir Sementsov-Ogievskiy >> >> Let's add a possibility to query dirty-bitmaps not only on root nodes. >> It is useful when dealing both with snapshots and incremental backups. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> [Added deprecation information. --js] >> Signed-off-by: John Snow >> --- >> block/qapi.c | 5 + >> qapi/block-core.json | 6 +- >> qemu-deprecated.texi | 12 >> 3 files changed, 22 insertions(+), 1 deletion(-) > >> +++ b/qapi/block-core.json >> @@ -360,6 +360,9 @@ >> # @write_threshold: configured write threshold for the device. >> # 0 if disabled. (Since 2.3) >> # >> +# @dirty-bitmaps: dirty bitmaps information (only present if node >> +# has one or more dirty bitmaps) (Since 4.2) >> +# > > Naming-wise, everything else in this struct uses 'foo_bar' while your > addition uses 'foo-bar'. But at this point, I don't know if it's worth > uglifying this addition just to fit in. > >> # Since: 0.14.0 >> # >> ## >> @@ -378,7 +381,7 @@ >> '*bps_wr_max_length': 'int', '*iops_max_length': 'int', >> '*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int', >> '*iops_size': 'int', '*group': 'str', 'cache': >> 'BlockdevCacheInfo', >> -'write_threshold': 'int' } } >> +'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo'] >> } } >> >> ## >> # @BlockDeviceIoStatus: >> @@ -656,6 +659,7 @@ >> # >> # @dirty-bitmaps: dirty bitmaps information (only present if the >> # driver has one or more dirty bitmaps) (Since 2.0) >> +# Deprecated in 4.2; see BlockDirtyInfo instead. > > s/BlockDirtyInfo/BlockDeviceInfo/ > > With the spelling fix, > Sigh, oops. > Reviewed-by: Eric Blake > > Is this worth squeezing into 4.1, to start the deprecation clock one > cycle earlier (on the grounds that the missing information for anonymous > nodes is a bug)? Or am I pushing the boundaries too far, where keeping > this as 4.2 material remains the best course of action? > Appealing option. If you think the deprecation plan is actionable enough for libvirt, I'm in favor. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/17/19 4:05 PM, Eric Blake wrote: > On 7/17/19 2:21 PM, John Snow wrote: >>> >>> Is this worth squeezing into 4.1, to start the deprecation clock one >>> cycle earlier (on the grounds that the missing information for anonymous >>> nodes is a bug)? Or am I pushing the boundaries too far, where keeping >>> this as 4.2 material remains the best course of action? >>> >> >> Appealing option. If you think the deprecation plan is actionable enough >> for libvirt, I'm in favor. > > I know my code for scraping query-block output during > virDomainCheckpointGetXMLDesc(,VIR_DOMAIN_CHECKPOINT_XML_SIZE) that > reports the size of the bitmap to the end user hasn't landed yet, and > that appears to be the only client in libvirt of this information at the > moment; but it's not a problem for me to check introspection for where > to find it (as libvirt already has a good framework for scraping > introspection for other reasons). > Ah, well... rc1 was yesterday already, so actually I think it's probably just really too late to do this. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/17/19 1:39 PM, John Snow wrote: > From: Vladimir Sementsov-Ogievskiy > > Let's add a possibility to query dirty-bitmaps not only on root nodes. > It is useful when dealing both with snapshots and incremental backups. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > [Added deprecation information. --js] > Signed-off-by: John Snow Made spelling edit suggested by Eric, and queued for 4.2. Thanks, applied to my bitmaps tree: https://github.com/jnsnow/qemu/commits/bitmaps https://github.com/jnsnow/qemu.git --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/18/19 6:20 AM, no-re...@patchew.org wrote: > PASS 17 test-bdrv-drain /bdrv-drain/graph-change/drain_all > = > ==10263==ERROR: AddressSanitizer: heap-use-after-free on address > 0x6122c1f0 at pc 0x555fd5bd7cb6 bp 0x7f3e853b8680 sp 0x7f3e853b8678 > WRITE of size 1 at 0x6122c1f0 thread T5 > ==10262==WARNING: ASan doesn't fully support makecontext/swapcontext > functions and may produce false positives in some cases! > #0 0x555fd5bd7cb5 in aio_notify /tmp/qemu-test/src/util/async.c:351:9 > #1 0x555fd5bd98eb in qemu_bh_schedule > /tmp/qemu-test/src/util/async.c:167:9 > #2 0x555fd5bdcaf0 in aio_co_schedule /tmp/qemu-test/src/util/async.c:464:5 I'm fairly certain that this isn't related to this patch. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/24/19 12:47 AM, Markus Armbruster wrote:
> John Snow writes:
>
>> From: Vladimir Sementsov-Ogievskiy
>>
>> Let's add a possibility to query dirty-bitmaps not only on root nodes.
>> It is useful when dealing both with snapshots and incremental backups.
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>> [Added deprecation information. --js]
>> Signed-off-by: John Snow
>> ---
>> block/qapi.c | 5 +
>> qapi/block-core.json | 6 +-
>> qemu-deprecated.texi | 12
>> 3 files changed, 22 insertions(+), 1 deletion(-)
>>
>> diff --git a/block/qapi.c b/block/qapi.c
>> index 917435f022..15f1030264 100644
>> --- a/block/qapi.c
>> +++ b/block/qapi.c
>> @@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend *blk,
>> info->backing_file = g_strdup(bs->backing_file);
>> }
>>
>> +if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
>> +info->has_dirty_bitmaps = true;
>> +info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
>> +}
>> +
>> info->detect_zeroes = bs->detect_zeroes;
>>
>> if (blk && blk_get_public(blk)->throttle_group_member.throttle_state) {
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 0d43d4f37c..9210ae233d 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -360,6 +360,9 @@
>> # @write_threshold: configured write threshold for the device.
>> # 0 if disabled. (Since 2.3)
>> #
>> +# @dirty-bitmaps: dirty bitmaps information (only present if node
>> +# has one or more dirty bitmaps) (Since 4.2)
>> +#
>> # Since: 0.14.0
>> #
>> ##
>> @@ -378,7 +381,7 @@
>> '*bps_wr_max_length': 'int', '*iops_max_length': 'int',
>> '*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int',
>> '*iops_size': 'int', '*group': 'str', 'cache':
>> 'BlockdevCacheInfo',
>> -'write_threshold': 'int' } }
>> +'write_threshold': 'int', '*dirty-bitmaps': ['BlockDirtyInfo']
>> } }
>>
>> ##
>> # @BlockDeviceIoStatus:
>> @@ -656,6 +659,7 @@
>> #
>> # @dirty-bitmaps: dirty bitmaps information (only present if the
>> # driver has one or more dirty bitmaps) (Since 2.0)
>> +# Deprecated in 4.2; see BlockDirtyInfo instead.
>> #
>> # @io-status: @BlockDeviceIoStatus. Only present if the device
>> # supports it and the VM is configured to stop on errors
>> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
>> index c90b08d553..6374b66546 100644
>> --- a/qemu-deprecated.texi
>> +++ b/qemu-deprecated.texi
>> @@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo''
>> structure, returned by
>> the query-block command is deprecated. Two new boolean fields,
>> ``recording'' and ``busy'' effectively replace it.
>>
>> +@subsection query-block result field dirty-bitmaps (Since 4.2)
>> +
>> +The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by
>> +the query-block command is itself now deprecated. The ``dirty-bitmaps``
>> +field of the ``BlockDeviceInfo`` struct should be used instead, which is the
>> +type of the ``inserted`` field in query-block replies, as well as the
>> +type of array items in query-named-block-nodes.
>
> Would the text be clearer if it talked only about commands, not about
> types?
>
> Here's my (laconic) try:
>
>@subsection query-block result field dirty-bitmaps (Since 4.2)
>
>In the result of query-block, member ``dirty-bitmaps'' has been moved
>into member ``inserted''.
>
Yeah, that's probably better in terms of strictly what the deprecation
is. I was trying to imply that the output will also now be visible in
other commands as well, but that's not the deprecation -- that's the new
feature.
ACK
> Aside: same for existing @subsection query-block result field
> dirty-bitmaps[i].status (since 4.0).
>
(Probably not worth editing deprecation text that was already published.)
>> +Since the ``dirty-bitmaps`` field is optionally present in both the old and
>> +new locations, clients must use introspection to learn where to anticipate
>> +t
Re: [libvirt] [Qemu-devel] [PATCH v3] qapi: add dirty-bitmaps to query-named-block-nodes result
On 7/25/19 2:06 AM, Markus Armbruster wrote:
> John Snow writes:
>
>> On 7/24/19 12:47 AM, Markus Armbruster wrote:
>>> John Snow writes:
>>>
>>>> From: Vladimir Sementsov-Ogievskiy
>>>>
>>>> Let's add a possibility to query dirty-bitmaps not only on root nodes.
>>>> It is useful when dealing both with snapshots and incremental backups.
>>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>> [Added deprecation information. --js]
>>>> Signed-off-by: John Snow
>>>> ---
>>>> block/qapi.c | 5 +
>>>> qapi/block-core.json | 6 +-
>>>> qemu-deprecated.texi | 12
>>>> 3 files changed, 22 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/block/qapi.c b/block/qapi.c
>>>> index 917435f022..15f1030264 100644
>>>> --- a/block/qapi.c
>>>> +++ b/block/qapi.c
>>>> @@ -79,6 +79,11 @@ BlockDeviceInfo *bdrv_block_device_info(BlockBackend
>>>> *blk,
>>>> info->backing_file = g_strdup(bs->backing_file);
>>>> }
>>>>
>>>> +if (!QLIST_EMPTY(&bs->dirty_bitmaps)) {
>>>> +info->has_dirty_bitmaps = true;
>>>> +info->dirty_bitmaps = bdrv_query_dirty_bitmaps(bs);
>>>> +}
>>>> +
>>>> info->detect_zeroes = bs->detect_zeroes;
>>>>
>>>> if (blk && blk_get_public(blk)->throttle_group_member.throttle_state)
>>>> {
>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>> index 0d43d4f37c..9210ae233d 100644
>>>> --- a/qapi/block-core.json
>>>> +++ b/qapi/block-core.json
>>>> @@ -360,6 +360,9 @@
>>>> # @write_threshold: configured write threshold for the device.
>>>> # 0 if disabled. (Since 2.3)
>>>> #
>>>> +# @dirty-bitmaps: dirty bitmaps information (only present if node
>>>> +# has one or more dirty bitmaps) (Since 4.2)
>>>> +#
>>>> # Since: 0.14.0
>>>> #
>>>> ##
>>>> @@ -378,7 +381,7 @@
>>>> '*bps_wr_max_length': 'int', '*iops_max_length': 'int',
>>>> '*iops_rd_max_length': 'int', '*iops_wr_max_length': 'int',
>>>> '*iops_size': 'int', '*group': 'str', 'cache':
>>>> 'BlockdevCacheInfo',
>>>> -'write_threshold': 'int' } }
>>>> +'write_threshold': 'int', '*dirty-bitmaps':
>>>> ['BlockDirtyInfo'] } }
>>>>
>>>> ##
>>>> # @BlockDeviceIoStatus:
>>>> @@ -656,6 +659,7 @@
>>>> #
>>>> # @dirty-bitmaps: dirty bitmaps information (only present if the
>>>> # driver has one or more dirty bitmaps) (Since 2.0)
>>>> +# Deprecated in 4.2; see BlockDirtyInfo instead.
>>>> #
>>>> # @io-status: @BlockDeviceIoStatus. Only present if the device
>>>> # supports it and the VM is configured to stop on errors
>>>> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
>>>> index c90b08d553..6374b66546 100644
>>>> --- a/qemu-deprecated.texi
>>>> +++ b/qemu-deprecated.texi
>>>> @@ -134,6 +134,18 @@ The ``status'' field of the ``BlockDirtyInfo''
>>>> structure, returned by
>>>> the query-block command is deprecated. Two new boolean fields,
>>>> ``recording'' and ``busy'' effectively replace it.
>>>>
>>>> +@subsection query-block result field dirty-bitmaps (Since 4.2)
>>>> +
>>>> +The ``dirty-bitmaps`` field of the ``BlockInfo`` structure, returned by
>>>> +the query-block command is itself now deprecated. The ``dirty-bitmaps``
>>>> +field of the ``BlockDeviceInfo`` struct should be used instead, which is
>>>> the
>>>> +type of the ``inserted`` field in query-block replies, as well as the
>>>> +type of array items in query-named-block-nodes.
>>>
>>> Would the text be clearer if it talked only about commands, not about
>>> types?
>>>
>>> Here's my (laconic) try:
>>>
&
Re: [libvirt] [Qemu-devel] [PATCH 1/2] qapi: deprecate drive-mirror and drive-backup
On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote:
> It's hard and not necessary to maintain outdated versions of these
> commands.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy
> ---
> qemu-deprecated.texi | 4
> qapi/block-core.json | 4
> qapi/transaction.json | 2 +-
> blockdev.c| 10 ++
> 4 files changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
> index fff07bb2a3..2753fafd0b 100644
> --- a/qemu-deprecated.texi
> +++ b/qemu-deprecated.texi
> @@ -179,6 +179,10 @@ and accurate ``query-qmp-schema'' command.
> Character devices creating sockets in client mode should not specify
> the 'wait' field, which is only applicable to sockets in server mode
>
> +@subsection drive-mirror, drive-backup and drive-backup transaction action
> (since 4.2)
> +
> +Use blockdev-mirror and blockdev-backup instead.
> +
> @section Human Monitor Protocol (HMP) commands
>
> @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove' (since
> 3.1)
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 0d43d4f37c..4e35526634 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1635,6 +1635,8 @@
> ##
> # @drive-backup:
> #
> +# Command is deprecated, use blockdev-mirror instead.
> +#
> # Start a point-in-time copy of a block device to a new destination. The
> # status of ongoing drive-backup operations can be checked with
> # query-block-jobs where the BlockJobInfo.type field has the value 'backup'.
> @@ -1855,6 +1857,8 @@
> ##
> # @drive-mirror:
> #
> +# Command is deprecated, use blockdev-mirror instead.
> +#
> # Start mirroring a block device's writes to a new destination. target
> # specifies the target of the new image. If the file exists, or if it
> # is a device, it will be used as the new destination for writes. If
> diff --git a/qapi/transaction.json b/qapi/transaction.json
> index 95edb78227..a16a9ff8a6 100644
> --- a/qapi/transaction.json
> +++ b/qapi/transaction.json
> @@ -53,7 +53,7 @@
> # - @blockdev-snapshot: since 2.5
> # - @blockdev-snapshot-internal-sync: since 1.7
> # - @blockdev-snapshot-sync: since 1.1
> -# - @drive-backup: since 1.6
> +# - @drive-backup: deprecated action, since 1.6
> #
> # Since: 1.1
> ##
> diff --git a/blockdev.c b/blockdev.c
> index 4d141e9a1f..36e9368e01 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1771,6 +1771,9 @@ static void drive_backup_prepare(BlkActionState
> *common, Error **errp)
> AioContext *aio_context;
> Error *local_err = NULL;
>
> +warn_report("drive-backup transaction action is deprecated and will "
> +"disappear in future. Use blockdev-backup action instead");
> +
> assert(common->action->type == TRANSACTION_ACTION_KIND_DRIVE_BACKUP);
> backup = common->action->u.drive_backup.data;
>
> @@ -3591,6 +3594,10 @@ void qmp_drive_backup(DriveBackup *arg, Error **errp)
> {
>
> BlockJob *job;
> +
> +warn_report("drive-backup command is deprecated and will disappear in "
> +"future. Use blockdev-backup instead");
> +
> job = do_drive_backup(arg, NULL, errp);
> if (job) {
> job_start(&job->job);
> @@ -3831,6 +3838,9 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
> const char *format = arg->format;
> int ret;
>
> +warn_report("drive-mirror command is deprecated and will disappear in "
> +"future. Use blockdev-mirror instead");
> +
> bs = qmp_get_root_bs(arg->device, errp);
> if (!bs) {
> return;
>
Hm!
I wonder if this is ever-so-slightly too soon for our friends over at
the libvirt project.
I don't think they have fully moved away from the non-blockdev
interfaces *just yet*, and I might encourage seeing the first full
libvirt release that does support and use it before we start the
deprecation clock.
(Jst in case.)
That's just me being very, very cautious though.
Peter Krempa, how do you feel about this?
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote:
> To get rid of implicit filters related workarounds in future let's
> deprecate them now.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy
> ---
> qemu-deprecated.texi | 7 +++
> qapi/block-core.json | 6 --
> include/block/block_int.h | 10 +-
> blockdev.c| 10 ++
> 4 files changed, 30 insertions(+), 3 deletions(-)
>
> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
> index 2753fafd0b..8222440148 100644
> --- a/qemu-deprecated.texi
> +++ b/qemu-deprecated.texi
> @@ -183,6 +183,13 @@ the 'wait' field, which is only applicable to sockets in
> server mode
>
> Use blockdev-mirror and blockdev-backup instead.
>
> +@subsection implicit filters (since 4.2)
> +
> +Mirror and commit jobs inserts filters, which becomes implicit if user
> +omitted filter-node-name parameter. So omitting it is deprecated, set it
> +always. Note, that drive-mirror don't have this parameter, so it will
> +create implicit filter anyway, but drive-mirror is deprecated itself too.
> +
> @section Human Monitor Protocol (HMP) commands
>
> @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove' (since
> 3.1)
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 4e35526634..0505ac9d8b 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1596,7 +1596,8 @@
> # @filter-node-name: the node name that should be assigned to the
> #filter driver that the commit job inserts into the graph
> #above @top. If this option is not given, a node name is
> -#autogenerated. (Since: 2.9)
> +#autogenerated. Omitting this option is deprecated, it
> will
> +#be required in future. (Since: 2.9)
> #
> # @auto-finalize: When false, this job will wait in a PENDING state after it
> has
> # finished its work, waiting for @block-job-finalize before
> @@ -2249,7 +2250,8 @@
> # @filter-node-name: the node name that should be assigned to the
> #filter driver that the mirror job inserts into the graph
> #above @device. If this option is not given, a node name
> is
> -#autogenerated. (Since: 2.9)
> +#autogenerated. Omitting this option is deprecated, it
> will
> +#be required in future. (Since: 2.9)
> #
> # @copy-mode: when to copy data to the destination; defaults to 'background'
> # (Since: 3.0)
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 3aa1e832a8..624da0b4a2 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -762,7 +762,15 @@ struct BlockDriverState {
> bool sg;/* if true, the device is a /dev/sg* */
> bool probed;/* if true, format was probed rather than specified */
> bool force_share; /* if true, always allow all shared permissions */
> -bool implicit; /* if true, this filter node was automatically inserted
> */
> +
> +/*
> + * @implicit field is deprecated, don't set it to true for new filters.
> + * If true, this filter node was automatically inserted and user don't
> + * know about it and unprepared for any effects of it. So, implicit
> + * filters are workarounded and skipped in many places of the block
> + * layer code.
> + */
> +bool implicit;
>
> BlockDriver *drv; /* NULL means no media */
> void *opaque;
> diff --git a/blockdev.c b/blockdev.c
> index 36e9368e01..b3cfaccce1 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -3292,6 +3292,11 @@ void qmp_block_commit(bool has_job_id, const char
> *job_id, const char *device,
> BlockdevOnError on_error = BLOCKDEV_ON_ERROR_REPORT;
> int job_flags = JOB_DEFAULT;
>
> +if (!has_filter_node_name) {
> +warn_report("Omitting filter-node-name parameter is deprecated, it "
> +"will be required in future");
> +}
> +
> if (!has_speed) {
> speed = 0;
> }
> @@ -3990,6 +3995,11 @@ void qmp_blockdev_mirror(bool has_job_id, const char
> *job_id,
> Error *local_err = NULL;
> int ret;
>
> +if (!has_filter_node_name) {
> +warn_report("Omitting filter-node-name parameter is deprecated, it "
> +"will be required in future");
> +}
> +
> bs = qmp_get_root_bs(device, errp);
> if (!bs) {
> return;
>
This might be OK to do right away, though.
I asked Markus this not too long ago; do we want to amend the QAPI
schema specification to allow commands to return with "Warning" strings,
or "Deprecated" stings to allow in-band deprecation notices for cases
like these?
example:
{ "return": {},
"deprecated": True,
"warning": "Omitting filter-node-name parameter is deprecated, it will
be required in the future"
}
There's no "error" key, so this should be re
Re: [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/15/19 6:49 AM, Kevin Wolf wrote:
> Am 14.08.2019 um 21:27 hat John Snow geschrieben:
>>
>>
>> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> To get rid of implicit filters related workarounds in future let's
>>> deprecate them now.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>> ---
>>> qemu-deprecated.texi | 7 +++
>>> qapi/block-core.json | 6 --
>>> include/block/block_int.h | 10 +-
>>> blockdev.c| 10 ++
>>> 4 files changed, 30 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
>>> index 2753fafd0b..8222440148 100644
>>> --- a/qemu-deprecated.texi
>>> +++ b/qemu-deprecated.texi
>>> @@ -183,6 +183,13 @@ the 'wait' field, which is only applicable to sockets
>>> in server mode
>>>
>>> Use blockdev-mirror and blockdev-backup instead.
>>>
>>> +@subsection implicit filters (since 4.2)
>>> +
>>> +Mirror and commit jobs inserts filters, which becomes implicit if user
>>> +omitted filter-node-name parameter. So omitting it is deprecated, set it
>>> +always. Note, that drive-mirror don't have this parameter, so it will
>>> +create implicit filter anyway, but drive-mirror is deprecated itself too.
>>> +
>>> @section Human Monitor Protocol (HMP) commands
>>>
>>> @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove'
>>> (since 3.1)
>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>> index 4e35526634..0505ac9d8b 100644
>>> --- a/qapi/block-core.json
>>> +++ b/qapi/block-core.json
>>> @@ -1596,7 +1596,8 @@
>>> # @filter-node-name: the node name that should be assigned to the
>>> #filter driver that the commit job inserts into the
>>> graph
>>> #above @top. If this option is not given, a node name
>>> is
>>> -#autogenerated. (Since: 2.9)
>>> +#autogenerated. Omitting this option is deprecated, it
>>> will
>>> +#be required in future. (Since: 2.9)
>>> #
>>> # @auto-finalize: When false, this job will wait in a PENDING state after
>>> it has
>>> # finished its work, waiting for @block-job-finalize before
>>> @@ -2249,7 +2250,8 @@
>>> # @filter-node-name: the node name that should be assigned to the
>>> #filter driver that the mirror job inserts into the
>>> graph
>>> #above @device. If this option is not given, a node
>>> name is
>>> -#autogenerated. (Since: 2.9)
>>> +#autogenerated. Omitting this option is deprecated, it
>>> will
>>> +#be required in future. (Since: 2.9)
>>> #
>>> # @copy-mode: when to copy data to the destination; defaults to
>>> 'background'
>>> # (Since: 3.0)
>>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>>> index 3aa1e832a8..624da0b4a2 100644
>>> --- a/include/block/block_int.h
>>> +++ b/include/block/block_int.h
>>> @@ -762,7 +762,15 @@ struct BlockDriverState {
>>> bool sg;/* if true, the device is a /dev/sg* */
>>> bool probed;/* if true, format was probed rather than specified */
>>> bool force_share; /* if true, always allow all shared permissions */
>>> -bool implicit; /* if true, this filter node was automatically
>>> inserted */
>>> +
>>> +/*
>>> + * @implicit field is deprecated, don't set it to true for new filters.
>>> + * If true, this filter node was automatically inserted and user don't
>>> + * know about it and unprepared for any effects of it. So, implicit
>>> + * filters are workarounded and skipped in many places of the block
>>> + * layer code.
>>> + */
>>> +bool implicit;
>>>
>>> BlockDriver *drv; /* NULL means no media */
>>> void *opaque;
>>> diff --git a/blockdev.c b/blockdev.c
>>> index 36e9368e01..b3cfaccce1 100644
>>> --- a/blockdev.c
>>> +++ b/blockdev.c
>>> @@ -3292,6 +3292,11 @@ void qmp_block_commit(bool has_job_id, const char
>>> *job_id, const char *device,
>>>
Re: [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/15/19 12:48 PM, Kevin Wolf wrote:
> Am 15.08.2019 um 18:07 hat John Snow geschrieben:
>> On 8/15/19 6:49 AM, Kevin Wolf wrote:
>>> Am 14.08.2019 um 21:27 hat John Snow geschrieben:
>>>>
>>>> This might be OK to do right away, though.
>>>>
>>>> I asked Markus this not too long ago; do we want to amend the QAPI
>>>> schema specification to allow commands to return with "Warning" strings,
>>>> or "Deprecated" stings to allow in-band deprecation notices for cases
>>>> like these?
>>>>
>>>> example:
>>>>
>>>> { "return": {},
>>>> "deprecated": True,
>>>> "warning": "Omitting filter-node-name parameter is deprecated, it will
>>>> be required in the future"
>>>> }
>>>>
>>>> There's no "error" key, so this should be recognized as success by
>>>> compatible clients, but they'll definitely see the extra information.
>>>>
>>>> Part of my motivation is to facilitate a more aggressive deprecation of
>>>> legacy features by ensuring that we are able to rigorously notify users
>>>> through any means that they need to adjust their scripts.
>>>
>>> Who would read this, though? In the best case it ends up deep in a
>>> libvirt log that nobody will look at because there was no error. In the
>>> more common case, the debug level is configured so that QMP traffic
>>> isn't even logged.
>>>
>>> Kevin
>>>
>>
>> I believe you are right, but I also can't shake the feeling that this
>> attitude ensures that we'll never find a way to expose this information
>> to the end-user. Is this not too defeatist?
>
> I think the discussed approach that seemed most likely to me to succeed
> was adding a command line option that makes QEMU just crash if you use a
> deprecated feature, and enable that in libvirt test cases (or possibly
> even any non-release builds, though maybe it's a bit harsh there).
>
>> I think deprecation notices in the QMP stream has two benefits:
>>
>> 1) Any direct usages via qmp-shell or manual JSON connection are likely
>> to see this message in development or testing. I feel the usage of QEMU
>> directly is more likely to increase with time as other stacks seek to
>> work around libvirt.
>>
>> [Whether or not they should is another question, but I believe the
>> current reality to be that people are trying to.]
>
> I don't know about other people, but as a human user, I don't care about
> deprecation notices. As long as something works, I use it, and once I
> get an error message back, I'll use something else.
>
> If I manually enter drive_mirror and get a warning back, that doesn't
> tell me that libvirt still does the same thing and needs to be fixed. It
> just tells me that in the future I might need to change the commands
> that I use manually.
>
That the message we return needs to be *useful* doesn't sound like a
count against it.
> I guess this would still prevent adding new libvirt features that build
> on deprecated QEMU features because some manual testing will be involved
> there. But was this ever a problem?
>
No, because until recently we didn't deprecate anything.
>> 2) Programmatic deprecation notices can't be presented to a user at all
>> if we don't send them; at least this way it becomes libvirt's problem
>> over what to do with them. Perhaps even just in testing and regression
>> suites libvirt can assert that it sees no deprecation warnings (or
>> whitelist certain ones it knows about.)
>>
>> In the case of libvirt, it's not even necessarily about making sure the
>> end user sees it, because it isn't even necessarily the user's fault --
>> it's libvirt's. This is a sure-fire programmatic way to communicate
>> compatibility changes to libvirt.
>
> If libvirt uses this to make test cases fail, it could work.
>
Yeah, I think there's solid use there. I'll continue along in Markus's
thread.
> Kevin
>
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] Exposing feature deprecation to machine clients (was: [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters)
On 8/15/19 10:16 AM, Markus Armbruster wrote:
> John Snow writes:
>
>> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> To get rid of implicit filters related workarounds in future let's
>>> deprecate them now.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>> ---
> [...]
>>> diff --git a/blockdev.c b/blockdev.c
>>> index 36e9368e01..b3cfaccce1 100644
>>> --- a/blockdev.c
>>> +++ b/blockdev.c
>>> @@ -3292,6 +3292,11 @@ void qmp_block_commit(bool has_job_id, const char
>>> *job_id, const char *device,
>>> BlockdevOnError on_error = BLOCKDEV_ON_ERROR_REPORT;
>>> int job_flags = JOB_DEFAULT;
>>>
>>> +if (!has_filter_node_name) {
>>> +warn_report("Omitting filter-node-name parameter is deprecated, it
>>> "
>>> +"will be required in future");
>>> +}
>>> +
>>> if (!has_speed) {
>>> speed = 0;
>>> }
>>> @@ -3990,6 +3995,11 @@ void qmp_blockdev_mirror(bool has_job_id, const char
>>> *job_id,
>>> Error *local_err = NULL;
>>> int ret;
>>>
>>> +if (!has_filter_node_name) {
>>> +warn_report("Omitting filter-node-name parameter is deprecated, it
>>> "
>>> +"will be required in future");
>>> +}
>>> +
>>> bs = qmp_get_root_bs(device, errp);
>>> if (!bs) {
>>> return;
>>>
>>
>> This might be OK to do right away, though.
>>
>> I asked Markus this not too long ago; do we want to amend the QAPI
>> schema specification to allow commands to return with "Warning" strings,
>> or "Deprecated" stings to allow in-band deprecation notices for cases
>> like these?
>>
>> example:
>>
>> { "return": {},
>> "deprecated": True,
>> "warning": "Omitting filter-node-name parameter is deprecated, it will
>> be required in the future"
>> }
>>
>> There's no "error" key, so this should be recognized as success by
>> compatible clients, but they'll definitely see the extra information.
>
> This is a compatible evolution of the QMP protocol.
>
>> Part of my motivation is to facilitate a more aggressive deprecation of
>> legacy features by ensuring that we are able to rigorously notify users
>> through any means that they need to adjust their scripts.
>
> Yes, we should help libvirt etc. with detecting use of deprecated
> features. We discussed this at the KVM Forum 2018 BoF on deprecating
> stuff. Minutes:
>
> Message-ID: <87mur0ls8o@dusky.pond.sub.org>
> https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html
>
> Last item is relevant here.
>
> Adding deprecation information to QMP's success response belongs to "We
> can also pass the buck to the next layer up", next to "emit a QMP
> event".
>
> Let's compare the two, i.e. "deprecation info in success response"
> vs. "deprecation event".
>
> 1. Possible triggers
>
> Anything we put in the success response should only ever apply to the
> (successful) command. So this one's limited to QMP commands.
>
> A QMP event is not limited to QMP commands. For instance, it could be
> emitted for deprecated CLI features (long after the fact, in addition to
> human-readable warnings on stderr), or when we detect use of a
> deprecated feature only after we sent the success response, say in a
> job. Neither use case is particularly convincing. Reporting use of
> deprecated CLI in QMP feels like a work-around for the CLI's
> machine-unfriendliness. Job-like commands should really check their
> arguments upfront.
>
> 2. Connection to trigger
>
> Connecting responses to commands is the QMP protocol's responsibility.
> Transmitting deprecation information in the response trivially ties it
> to the offending command.
>
> The QMP protocol doesn't tie events to anything. Thus, a deprecation
> event needs an event-specific tie to its trigger.
>
> The obvious way to tie it to a command mirrors how the QMP protocol ties
> responses to commands: by command ID. The event either has to be sent
> just to the offending monitor (currently, all events are broadcast to
> all monitors), or include a suitable monitor ID.
>
> For non-command triggers, we'd have to invent some other tie.
>
Re: [libvirt] [Qemu-devel] [PATCH 1/2] qapi: deprecate drive-mirror and drive-backup
On 8/15/19 3:44 AM, Peter Krempa wrote:
> On Wed, Aug 14, 2019 at 15:22:15 -0400, John Snow wrote:
>>
>>
>> On 8/14/19 6:07 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> It's hard and not necessary to maintain outdated versions of these
>>> commands.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>> ---
>>> qemu-deprecated.texi | 4
>>> qapi/block-core.json | 4
>>> qapi/transaction.json | 2 +-
>>> blockdev.c| 10 ++
>>> 4 files changed, 19 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
>>> index fff07bb2a3..2753fafd0b 100644
>>> --- a/qemu-deprecated.texi
>>> +++ b/qemu-deprecated.texi
>>> @@ -179,6 +179,10 @@ and accurate ``query-qmp-schema'' command.
>>> Character devices creating sockets in client mode should not specify
>>> the 'wait' field, which is only applicable to sockets in server mode
>>>
>>> +@subsection drive-mirror, drive-backup and drive-backup transaction action
>>> (since 4.2)
>>> +
>>> +Use blockdev-mirror and blockdev-backup instead.
>>> +
>>> @section Human Monitor Protocol (HMP) commands
>>>
>>> @subsection The hub_id parameter of 'hostfwd_add' / 'hostfwd_remove'
>>> (since 3.1)
>
> [...]
>
>>> @@ -3831,6 +3838,9 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
>>> const char *format = arg->format;
>>> int ret;
>>>
>>> +warn_report("drive-mirror command is deprecated and will disappear in "
>>> +"future. Use blockdev-mirror instead");
>>> +
>>> bs = qmp_get_root_bs(arg->device, errp);
>>> if (!bs) {
>>> return;
>>>
>>
>> Hm!
>>
>> I wonder if this is ever-so-slightly too soon for our friends over at
>> the libvirt project.
>>
>> I don't think they have fully moved away from the non-blockdev
>> interfaces *just yet*, and I might encourage seeing the first full
>> libvirt release that does support and use it before we start the
>> deprecation clock.
>>
>> (Jst in case.)
>>
>> That's just me being very, very cautious though.
>>
>> Peter Krempa, how do you feel about this?
>
> Thanks for the heads up!
>
You're welcome!
> Currently libvirt does not use 'drive-backup' at all so that one can be
> deprecated immediately.
>
> In case of 'drive-mirror' the situation is a bit more complex:
>
> Libvirt uses 'drive-mirror' currently in the following places
>
> 1) virDomainBlockCopy API
> With blockdev integration enabled this will go away. Pathces are being
> reviewed:
>
> https://www.redhat.com/archives/libvir-list/2019-August/msg00295.html
>
> 2) VM migration with non-shared storage
> Currently uses 'drive-mirror' in most cases but there is pre-existing
> integration for blockdev-mirror for nbd+tls. I need to make sure that
> the blockdev version will be used unconditionally once the integration
> is enabled. This is a TODO.
>
> There is also one gotcha. In case when an 'sd' card device is used for
> the VM, libvirt disables all of blockdev, because SD cards can't be
> expressed with blockdev. There's too many code paths which would need
> checking to be worth it. To be fair, I'm not even sure when a sd card
> can be emulated by qemu as all of my basic tests failed and I did not
> care more.
>
> For libvirt to enable blockdev there's one more part missing and that's
> snapshot integration. I'm currently testing patches to integrate it with
> external snapshots, which should be posted soon.
>
> I also found a bug in qemu, which prevents creation of internal
> snapshots when -blockdev is used:
>
> When savevm HMP command is used (via QMP->HMP bridge) qemu invokes
> save_snapshot(), which calls bdrv_all_can_snapshot(). That function uses
> bdrv_next() to iterate all nodes which correspond to a block backend
> first, but then also iterates any other node which is monitor-owned.
>
> Since with blockdev all nodes including the ones for the 'file' protocol
> are monitor-owned, and 'file' does not support snapshots that check
> fails. A simple hack of skipping the second part in bdrv_next() allows
> to do a snapshot actually. Kevin told me that the idea is that also
> non-attached nodes should be considered for internal snapshod which is
> okay in my opinion, but given how the snapshot works for the files
> attached to backeds (and also in pre-blockdev use) only the top level of
> a chain should ever be considered for snapshot.
>
> So the summary is, that I'm pretty hopeful that we should be able to get
> rid of all reasonable uses of drive-mirror very soon after I finish
> snapshot integration. The only question is how much
> we care about SD card users being able to do a drive-mirror in the
> future.
>
OK. It sounds like we should hold off on deprecating these for now
because it's not certain which libvirt release will no longer need them,
but it sounds like it's hopefully not far off.
--js
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] QEMU bitmap backup usability FAQ
Hi, downstream here at Red Hat I've been fielding some questions about the usability and feature readiness of Bitmaps (and related features) in QEMU. Here are some questions I answered internally that I am copying to the list for two reasons: (1) To make sure my answers are actually correct, and (2) To share this pseudo-reference with the community at large. This is long, and mostly for reference. There's a summary at the bottom with some todo items and observations about the usability of the feature as it exists in QEMU. Before too long, I intend to send a more summarized "roadmap" mail which details all of the current and remaining work to be done in and around the bitmaps feature in QEMU. Questions: > "What format(s) is/are required for this functionality?" >From the QEMU API, any format can be used to create and author incremental backups. The only known format limitations are: 1. Persistent bitmaps cannot be created on any format except qcow2, although there are hooks to add support to other formats at a later date if desired. DANGER CAVEAT #1: Adding bitmaps to QEMU by default creates transient bitmaps instead of persistent ones. Possible TODO: Allow users to 'upgrade' transient bitmaps to persistent ones in case they made a mistake. 2. When using push backups (blockdev-backup, drive-backup), you may use any format as a target format. DANGER CAVEAT #2: without backing file and/or filesystem-less sparse support, these images will be unusable. EXAMPLE: Backing up to a raw file loses allocation information, so we can no longer distinguish between zeroes and unallocated regions. The cluster size is also lost. This file will not be usable without additional metadata recorded elsewhere.* (* This is complicated, but it is in theory possible to do a push backup to e.g. an NBD target with custom server code that saves allocation information to a metadata file, which would allow you to reconstruct backups. For instance, recording in a .json file which extents were written out would allow you to -- with a custom binary -- write this information on top of a base file to reconstruct a backup.) 3. Any format can be used for either shared storage or live storage migrations. There are TWO distinct mechanisms for migrating bitmaps: A) The bitmap is flushed to storage and re-opened on the destination. This is only supported for qcow2 and shared-storage migrations. B) The bitmap is live-migrated to the destination. This is supported for any format and can be used for either shared storage or live storage migrations. DANGER CAVEAT #3: The second bitmap migration technique there is an optional migration capability that must be enabled explicitly. Otherwise, some migration combinations may drop bitmaps. Matrix: > migrate = migrate_capability or (persistent and shared_storage) Enumerated: live storage + raw : transient + no-capability: Dropped live-storage + raw : transient + bm-capability: Migrated live-storage + qcow2 : transient + no-capability: Dropped live-storage + qcow2 : transient + bm-capability: Migrated live-storage + qcow2 : persistent + no-capability: Dropped (!) live-storage + qcow2 : persistent + bm-capability: Migrated shared-storage + raw : transient - no-capability: Dropped shared-storage + raw : transient + bm-capability: Migrated shared-storage + qcow2 : transient + no-capability: Migrated shared-storage + qcow2 : transient + bm-capability: Migrated shared-storage + qcow2 : persistent + no-capability: Migrated shared-storage + qcow2 : persistent + bm-capability: Migrated Enabling the bitmap migration capability will ALWAYS migrate the bitmap. If it's disabled, we will only migrate the bitmaps for shared storage migrations where the bitmap is persistent, which is a qcow2-only case. There is no warning or error if you attempt to migrate in a manner that loses your bitmaps. (I might be persuaded to add a case for when you are doing a live storage migration of qcow2 with persistent bitmaps, which is somewhat a conflicting case: you've asked for the bitmap to be persistent, but it seems likely that if this ever happens in practice, it's because you have neglected to ask for it to be migrated to the new host.) See iotest 169 for more details on this. At present, these are the only format limitations I am consciously aware of. From a management API/GUI perspective, it makes sense to restrict the feature set to "qcow2 only" to minimize edge cases. > "Is libvirt aware of these 'gotcha' cases?" >From talks I've had with Eric Blake and Peter Krempa, they certainly are now. > "Is it possible to make persistent the default?" Not quickly. In QEMU, not without a deprecation period or some other incompatibility. Default values are not (yet?) introspectable via the schema. We need (possibly) up to two QAPI extensions: I) The ability to return deprecation warnings when issuing a command that will cease to work in the future. This has been being discussed somewhat on-list recently.
Re: [libvirt] QEMU bitmap backup usability FAQ
On 8/21/19 10:21 AM, Vladimir Sementsov-Ogievskiy wrote: > [CC Nikolay] > > 21.08.2019 1:25, John Snow wrote: >> Hi, downstream here at Red Hat I've been fielding some questions about >> the usability and feature readiness of Bitmaps (and related features) in >> QEMU. >> >> Here are some questions I answered internally that I am copying to the >> list for two reasons: >> >> (1) To make sure my answers are actually correct, and >> (2) To share this pseudo-reference with the community at large. >> >> This is long, and mostly for reference. There's a summary at the bottom >> with some todo items and observations about the usability of the feature >> as it exists in QEMU. >> >> Before too long, I intend to send a more summarized "roadmap" mail which >> details all of the current and remaining work to be done in and around >> the bitmaps feature in QEMU. >> >> >> Questions: >> >>> "What format(s) is/are required for this functionality?" >> >> From the QEMU API, any format can be used to create and author >> incremental backups. The only known format limitations are: >> >> 1. Persistent bitmaps cannot be created on any format except qcow2, >> although there are hooks to add support to other formats at a later date >> if desired. >> >> DANGER CAVEAT #1: Adding bitmaps to QEMU by default creates transient >> bitmaps instead of persistent ones. >> >> Possible TODO: Allow users to 'upgrade' transient bitmaps to persistent >> ones in case they made a mistake. > > I doubt, as in my opinion real users of Qemu are not people but libvirt, which > should never make such mistake. > Right, that's largely been the consensus here; but there is some concern that libvirt might not be the only user of QEMU, so I felt it was worth documenting some of the crucial moments where it was "easy" to get it wrong. I think like it or not, the API that QEMU presents has to be considered on its own without libvirt because it's not a given that libvirt will forever and always be the only user of QEMU. I do think that any problems of this kind that can be solved in libvirt are not immediate, crucial action items. libvirt WILL be the major user of these features. However, try as we might, releasing a set of primitive operations that offer 998 ways to corrupt your data and 2 ways to manage it correctly are going to provoke some questions from people who are trying to work with that API, including from libvirt developers. It might be the conclusion that it's libvirt's job to safeguard the user from themselves, but we at least need to present consistent and clear information about the way we expect/anticipate people to use the APIs, because people DO keep asking me about several of these issues and the usability problems they perceive with the QEMU API. So this thread was largely in attempt to explore what some "solutions" to perceived problems look like, mostly to come to the conclusion that the actual "must-haves" list in QEMU is not very long compared to the "nice-to-haves?" list. >> >> >> 2. When using push backups (blockdev-backup, drive-backup), you may use >> any format as a target format. > >> DANGER CAVEAT #2: without backing file and/or filesystem-less sparse >> support, these images will be unusable. > > You mean incremental backups of course, as the whole document is about > bitmaps. > Ah, yes, incremental push backups. Full backups are of course not a problem. :) >> >> EXAMPLE: Backing up to a raw file loses allocation information, so we >> can no longer distinguish between zeroes and unallocated regions. The >> cluster size is also lost. This file will not be usable without >> additional metadata recorded elsewhere.* >> >> (* This is complicated, but it is in theory possible to do a push backup >> to e.g. an NBD target with custom server code that saves allocation >> information to a metadata file, which would allow you to reconstruct >> backups. For instance, recording in a .json file which extents were >> written out would allow you to -- with a custom binary -- write this >> information on top of a base file to reconstruct a backup.) >> >> >> 3. Any format can be used for either shared storage or live storage >> migrations. There are TWO distinct mechanisms for migrating bitmaps: >> >> A) The bitmap is flushed to storage and re-opened on the destination. >> This is only supported for qcow2 and shared-storage migrations. > > cons: flushing/reopening is done during migration downtime, so if you have > a
Re: [libvirt] [PATCH 2/2] qapi: deprecate implicit filters
On 8/23/19 5:22 AM, Vladimir Sementsov-Ogievskiy wrote: > 14.08.2019 13:07, Vladimir Sementsov-Ogievskiy wrote: >> To get rid of implicit filters related workarounds in future let's >> deprecate them now. > > Interesting, could we deprecate implicit filter without deprecation of > unnecessity of > parameter? As actually, it's good when this parameter is not necessary, in > most cases > user is not interested in node-name. > https://en.wiktionary.org/wiki/unnecessity -- I am surprised to learn that this a real word in the language I speak. :) I assume you're referring to making the optional argument mandatory. > Obviously we can do the following: > > 1. In 4.2 we deprecate unnecessity, which implies deprecation of implicit > filters > 2. After some releases in 4.x we can drop deprecated functionality, so we > drop it together with > implicit filters. And, in same release 4.x we return it back (as it's > compatible change :) > but without implicit filters (so, if filter-node-name not specified, we just > create > explicit filter with autogenerated node-name) > > So, effectively we just drop "deprecation mark" together with implicit > filters, which is nice > but actually confusing. > > Instead, we may do > 1. In 4.2 deprecate > 2. In 4.x drop optionality together with implicit filters > 3. In 4.y (y > x of course) return optionality back > Ah, I see what you're digging at here now... > It's a bit safer, but for users who miss releases [4.x, 4.y) it's no > difference.. > > Or we just write in spec, that implicit filters are deprecated? But we have > nothing about implicit > filters in spec. More over, we directly write that we have filter, and if > parameter is omitted > it's node-name is autogenerated. So actually, the fact the filter is hidden > when filter-node-name is > unspecified is _undocumented_. > > So, finally, it looks like nothing to deprecated in specification, we can > just drop implicit filters :) > > What do you think? > What exactly _IS_ an implicit filter? How does it differ today from an explicit filter? I assumed the only difference was if it was named or not; but I think I must be mistaken now if you're proposing leaving the interface alone entirely. Are they instantiated differently? --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 2/2] qapi: deprecate implicit filters
(Peter: search for "pkrempa" down below.)
On 8/28/19 5:20 AM, Vladimir Sementsov-Ogievskiy wrote:
> 27.08.2019 23:12, John Snow wrote:
>>
>>
>> On 8/23/19 5:22 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> 14.08.2019 13:07, Vladimir Sementsov-Ogievskiy wrote:
>>>> To get rid of implicit filters related workarounds in future let's
>>>> deprecate them now.
>>>
>>> Interesting, could we deprecate implicit filter without deprecation of
>>> unnecessity of
>>> parameter? As actually, it's good when this parameter is not necessary, in
>>> most cases
>>> user is not interested in node-name.
>>>
>>
>> https://en.wiktionary.org/wiki/unnecessity -- I am surprised to learn
>> that this a real word in the language I speak. :)
>>
>> I assume you're referring to making the optional argument mandatory.
>
> exactly, it's my a bit "google-translate-driven" English)
>
It teaches me some fun words!
>>
>>> Obviously we can do the following:
>>>
>>> 1. In 4.2 we deprecate unnecessity, which implies deprecation of implicit
>>> filters
>>> 2. After some releases in 4.x we can drop deprecated functionality, so we
>>> drop it together with
>>> implicit filters. And, in same release 4.x we return it back (as it's
>>> compatible change :)
>>> but without implicit filters (so, if filter-node-name not specified, we
>>> just create
>>> explicit filter with autogenerated node-name)
>>>
>>> So, effectively we just drop "deprecation mark" together with implicit
>>> filters, which is nice
>>> but actually confusing.
>>>
>>> Instead, we may do
>>> 1. In 4.2 deprecate
>>> 2. In 4.x drop optionality together with implicit filters
>>> 3. In 4.y (y > x of course) return optionality back
>>>
>>
>> Ah, I see what you're digging at here now...
>>
>>> It's a bit safer, but for users who miss releases [4.x, 4.y) it's no
>>> difference..
>>>
>>> Or we just write in spec, that implicit filters are deprecated? But we have
>>> nothing about implicit
>>> filters in spec. More over, we directly write that we have filter, and if
>>> parameter is omitted
>>> it's node-name is autogenerated. So actually, the fact the filter is hidden
>>> when filter-node-name is
>>> unspecified is _undocumented_.
>>>
>>> So, finally, it looks like nothing to deprecated in specification, we can
>>> just drop implicit filters :)
>>>
>>> What do you think?
>>>
>>
>> What exactly _IS_ an implicit filter? How does it differ today from an
>> explicit filter? I assumed the only difference was if it was named or
>> not; but I think I must be mistaken now if you're proposing leaving the
>> interface alone entirely.
>>
>> Are they instantiated differently?
>>
>
> As I understand, the only difference is their BlockDriverState.impicit field,
> and several places in code
> where we skip implicit filter when trying to find something in a chain
> starting from a device.
>
Oh, oh, yes. I see.
> Hmm, OK, let's see:
>
> 1. the only implicit filters are commit_top and mirror_top if user don't
> specify filter-node-name.
>
> Where it make sense, i.e., where implicit field used?
>
`git grep -E '(->|\.)implicit'` is what I used to find usages.
> 2. bdrv_query_info, bdrv_query_bds_stats, bdrv_block_device_info(only when
> called from bdrv_query_info), they'll
> report filter as top node if we don't mark it implicit.
>
So that's a bit of a change, but only visually. The "reality" is still
the same, we just report it more "accurately." libvirt MIGHT need a
heads up here. I'm looping pkrempa back in for comment.
Would libvirt be negatively impacted by the revelation of formerly
internal ("implicit") nodes created by mirror and commit via query block
commands? At the moment, QEMU hides them from you if you do not name them.
> 3. bdrv_refresh_filename, bdrv_reopen_parse_backing, bdrv_drop_intermediate:
>I think it's not a problem, just drop special case for implicit fitlers
>
I'm much less certain about what the impact of this would be and would
need to audit it (and don't have the time to, personally.)
Do you have a POC or RFC patch that demonstrates dropping these special
cases? It might be nice to see as proof that it's safe to deprecate.
> So, seems the only real change is
Re: [libvirt] [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/29/19 11:59 AM, Christophe de Dinechin wrote:
>
> John Snow writes:
> [...]
>>
>> This might be OK to do right away, though.
>>
>> I asked Markus this not too long ago; do we want to amend the QAPI
>> schema specification to allow commands to return with "Warning" strings,
>> or "Deprecated" stings to allow in-band deprecation notices for cases
>> like these?
>>
>> example:
>>
>> { "return": {},
>> "deprecated": True,
>> "warning": "Omitting filter-node-name parameter is deprecated, it will
>> be required in the future"
>> }
>>
>> There's no "error" key, so this should be recognized as success by
>> compatible clients, but they'll definitely see the extra information.
>>
>> Part of my motivation is to facilitate a more aggressive deprecation of
>> legacy features by ensuring that we are able to rigorously notify users
>> through any means that they need to adjust their scripts.
>
> I like this approach even if there is no consumer today. It does not
> hurt, and it is indeed a motivation to develop consumers that care.
>
> I personally find this much easier to swallow than any kind of crash on
> deprecation, which already at the BoF seemed like a really big hammer to
> kill a fly.
>
> CC'ing Andrea as well, because we discussed recently about how to deal
> with error checking in general, and if a new error checking framework is
> being put in place, adding deprecation to the thinking could be a good
> idea.
The most convincing argument against deprecation notices like this is
not that they won't be consumed, but that they are difficult to plumb
through the C infrastructure.
Sadly, I think I have to agree there -- we can't even really model it
like hints, because these are cases where there was no /error/ but
instead a success -- but our error propagation doesn't work on those
terms generally and we'd need a rather extensive audit to allow warnings.
We could always fudge it with a kind of global warning log: clear the
log at the beginning of a QMP interaction and if the log is non-empty
when we return, amend the return with that information.
That's not really the nicest thing to do in a multi-process,
multi-threaded, multi-stacked application, though, so...
--js
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 2/2] qapi: deprecate implicit filters
On 8/29/19 11:17 AM, Vladimir Sementsov-Ogievskiy wrote: > Aren't you going to deprecate and drop it at some moment? Libvirt's going to keep supporting older versions of QEMU in perpetuity. Apparently, there are still some barriers (SD cards?) to using only blockdev API for new versions, too: >> "Note that even with new qemu, if an SD card is used blockdev will be >> disabled." It sounds like we need to facilitate libvirt's transfer to an all-blockdev API for modern QEMU instances. Once we do that, we can likely add an introspectable feature flag to commands that create formerly-implicit nodes to tip off libvirt as to what behavior it can expect. (In practice, if libvirt sees the new flag, it knows it needs to rely exclusively on blockdev API and that (likely) it should always provide a node-name for the filter.) I think it is reasonably clear that deprecating and re-implementing is not something that will be compatible with libvirt's feature detection, so we shouldn't do it. I'm still not sold that this is worth the effort, but you and Max would know best right now. I'll leave it to you two to sort out. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/29/19 12:45 PM, Christophe de Dinechin wrote: > > Markus Armbruster writes: > >> Peter Krempa writes: >> > [...] >>> From my experience users report non-fatal messages mostly only if it is >>> spamming the system log. One of instances are very unlikely to be >>> noticed. >>> >>> In my experience it's better to notify us in libvirt of such change and >>> we will try our best to fix it. >> >> How to best alert the layers above QEMU was one of the topic of the KVM >> Forum 2018 BoF on deprecating stuff. Minutes: >> >> Message-ID: <87mur0ls8o@dusky.pond.sub.org> >> https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg05828.html >> >> Relevant part: >> >> * We need to communicate "you're using something that is deprecated". >> How? Right now, we print a deprecation message. Okay when humans use >> QEMU directly in a shell. However, when QEMU sits at the bottom of a >> software stack, the message will likely end up in a log file that is >> effectively write-only. >> >> - The one way to get people read log files is crashing their >> application. A command line option --future could make QEMU crash >> right after printing a deprecation message. This could help with >> finding use of deprecated features in a testing environment. >> >> - A less destructive way to grab people's attention is to make things >> run really, really slow: have QEMU go to sleep for a while after >> printing a deprecation message. >> >> - We can also pass the buck to the next layer up: emit a QMP event. >> >> Sadly, by the time the next layer connects to QMP, plenty of stuff >> already happened. We'd have to buffer deprecation events somehow. >> >> What would libvirt do with such an event? Log it, taint the domain, >> emit a (libvirt) event to pass it on to the next layer up. >> >> - A completely different idea is to have a configuratin linter. To >> support doing this at the libvirt level, QEMU could expose "is >> deprecated" in interface introspection. Feels feasible for QMP, >> where we already have sufficiently expressive introspection. For >> CLI, we'd first have to provide that (but we want that anyway). >> >> - We might also want to dispay deprecation messages in QEMU's GUI >> somehow, or on serial consoles. > > Sorry for catching up late, this mail thread happened during my PTO. > > I remember bringing up at the time [1] that the correct solution needs > to take into account usage models that vary from > > - a workstation case, where displaying an error box is easy and > convenient, > > - to local headless VMs where system-level notification would do the job > better, allowing us to leverage things like system-wide email notifications > > - to large-scale collections of VMs managed by some layered product, > where the correct reporting would be through something like Insights, > i.e. you don't scan individual logs, you want something like "913 VMs > are using deprecated X" > > To me, that implies that we need to have a clear division of roles, with > a standard way to > > a) produce the errors, > b) propagate them, I started replying to this thread to the other mail you sent; I think this is going to be fairly involved. I wouldn't mind being proven wrong though. > c) consume them (at least up to libvirt) > > Notice that this work has already been done for "real" errors, > i.e. there is a real QAPI notion of "errors". AFAICT, warn_report does > not connect to it, though, it goes through error_vprintf which is really > just basic logging. > > So would it make sense to: > > 1. Add a deprecation_report() alongside warn_report()? > Where's that get routed to? just an error_vprintf style situation? > 2. Connect warn_report() and all the error_vprintf output to QAPI, >e.g. using John's suggestion of adding the messages using some >"warning" or "deprecated" tag? > How do you correlate them? > 3. Teach libvirt how to consume that new tag and pass it along? > I think it's not libvirt's job to pass it along, exactly -- libvirt made the decision for which features to engage in QEMU, not the end user. If the user upgrades QEMU but not libvirt, it's not really anything they have control over and they shouldn't be pestered with such things. However, if libvirt accidentally released a version that engages deprecated behavior (and were unaware of it), it'd be nice to get user reports, surely? Logging messages for libvirt might be the best that can be done there in that case. In contrast, power user tools like QMP libraries, qmp-shell and others allow more direct and meaningful access to QMP, so those should report deprecation messages to the user. > > [1] https://lists.nongnu.org/archive/html/qemu-devel/2018-10/msg06131.html > > > -- > Cheers, > Christophe de Dinechin (IRC c3d) > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters
On 8/30/19 6:07 AM, Christophe de Dinechin wrote:
>
> John Snow writes:
>
>> On 8/29/19 12:45 PM, Christophe de Dinechin wrote:
>>>
> [...]
>
>>> Sorry for catching up late, this mail thread happened during my PTO.
>>>
>>> I remember bringing up at the time [1] that the correct solution needs
>>> to take into account usage models that vary from
>>>
>>> - a workstation case, where displaying an error box is easy and
>>> convenient,
>>>
>>> - to local headless VMs where system-level notification would do the job
>>> better, allowing us to leverage things like system-wide email
>>> notifications
>>>
>>> - to large-scale collections of VMs managed by some layered product,
>>> where the correct reporting would be through something like Insights,
>>> i.e. you don't scan individual logs, you want something like "913 VMs
>>> are using deprecated X"
>>>
>>> To me, that implies that we need to have a clear division of roles, with
>>> a standard way to
>>>
>>> a) produce the errors,
>>> b) propagate them,
>>
>> I started replying to this thread to the other mail you sent; I think
>> this is going to be fairly involved. I wouldn't mind being proven wrong
>> though.
>
> Yes, I think it does look involved, but mostly for historical reasons.
> In other words, what is complicated is preserving the historical
> behaviors so as to not break existing consumers.
>
>>
>>> c) consume them (at least up to libvirt)
>>>
>>> Notice that this work has already been done for "real" errors,
>>> i.e. there is a real QAPI notion of "errors". AFAICT, warn_report does
>>> not connect to it, though, it goes through error_vprintf which is really
>>> just basic logging.
>>>
>>> So would it make sense to:
>>>
>>> 1. Add a deprecation_report() alongside warn_report()?
>>>
>>
>> Where's that get routed to? just an error_vprintf style situation?
>
> Yes, but see below.
>
>>
>>> 2. Connect warn_report() and all the error_vprintf output to QAPI,
>>>e.g. using John's suggestion of adding the messages using some
>>>"warning" or "deprecated" tag?
>>>
>>
>> How do you correlate them?
>
> Without having looked at the code much, I think I would
>
> 1. extend the existing QAPI error to support warnings, deprecations and
>info messages. The first problem I see is that there is no error, so
>we may sometimes need to create one when there was none before. And
>of course make sure that this does not ultimately show as an error,
>but as a success with additional annotations.
>
I assume this might be a chance to consolidate all of the methodologies
we use for actually checking if there was an error or not. There have
been many and I am sure Markus can give us a history lesson if it's
warranted.
Generally, there's a few paradigms I see a lot:
1. Rely on an error return code being produced by the called function.
The caller trusts that errp was set. This is one of my favorite methods,
because it has the least scaffolding.
2. Pass errp directly to the called function, and check for null after
return. I don't like this method very much, because of confusion with:
3. Create a local error object; check THAT for null, and propagate the
error to the common error object. I think Markus has explained why we
have this code 50 times, and I forget again minutes later.
If we want to expand the concept of the error object into something that
encompasses hints, warnings and deprecations*, checking for null is no
longer appropriate. It might be a good chance to make our error
propagation story more consistent, too.
We could unify with a helper like this, I think, if I'm not forgetting
some crucial usage detail:
subroutine(foo, bar, errp);
if (failure(errp)) {
error_append_hint(errp, "Lorem ipsum, ...");
cleanup();
return;
}
We would then always use this pattern that operates directly on the
caller's errp instead of creating local error objects to allow hints and
warnings to accumulate.
(* I'm not proposing all three in a concrete way, but just referencing
the general class of semantic non-error information we may want to
propagate, however we end up deciding to model it.)
> 2. replace the current "link + if" switching for error_vprintf with some
>actual notification mechanism, with one option routine to
>monitor_vprintf, one to stderr, one to log file, and then an
>additional path that would po
Re: [libvirt] [Qemu-block] [PATCH] docs: Update preferred NBD device syntax
On 9/3/19 3:02 PM, Eric Blake wrote: > [adding libvirt list] > > On 9/3/19 1:50 PM, John Snow wrote: >> >> >> On 9/3/19 10:56 AM, Eric Blake wrote: >>> Mention the preferred URI form, especially since NBD is trying to >>> standardize that form: https://lists.debian.org/nbd/2019/06/msg00012.html >>> >>> Signed-off-by: Eric Blake >>> --- >>> qemu-doc.texi | 16 +++- >>> 1 file changed, 11 insertions(+), 5 deletions(-) >>> >>> diff --git a/qemu-doc.texi b/qemu-doc.texi >>> index 577d1e837640..c83fb347d77e 100644 >>> --- a/qemu-doc.texi >>> +++ b/qemu-doc.texi >>> @@ -297,7 +297,14 @@ qemu-system-i386 -drive >>> file=iscsi://192.0.2.1/iqn.2001-04.com.example/1 >>> >>> @item NBD >>> QEMU supports NBD (Network Block Devices) both using TCP protocol as well >>> -as Unix Domain Sockets. >>> +as Unix Domain Sockets. With TCP, the default port is 10809. >>> >>> -Syntax for specifying a NBD device using TCP >>> +Syntax for specifying a NBD device using TCP, in preferred URI form: >>> +``nbd://[:]/[]'' >>> + >>> +Syntax for specifying a NBD device using Unix Domain Sockets; remember >>> +that '?' is a shell glob character and may need quoting: >>> +``nbd+unix:///[]?socket='' >>> + >>> +Older syntax that is also recognized: >> >> Deprecated officially, or no? >> >>> ``nbd::[:exportname=]'' >>> >>> -Syntax for specifying a NBD device using Unix Domain Sockets >>> ``nbd:unix:[:exportname=]'' > > I didn't feel like starting a deprecation clock, in part because libvirt > is still using nbd:host:port:exportname during migration, similarly code > in virstoragefile.c is using only the old form. Do we want to start a > deprecation (as a separate patch), to prod faster changes in libvirt in > switching to the newer form where sensible? > Yeah, understood -- I was merely curious for wording purposes. Some people might wonder what "Older syntax" means and perhaps why they shouldn't use it. It sounds like we do want to wander away from it eventually but aren't prepared to do that yet. I think largely such a deprecation clock is up to the workload of whoever would have to update the libvirt workflow (You, Peter?) and how much benefit we'd gain by dropping it in QEMU (little?) If you don't have motivation for doing it unprompted I have little reason to coerce you into it. >>> >>> Example for TCP >>> @example >>> -qemu-system-i386 --drive file=nbd:192.0.2.1:3 >>> +qemu-system-i386 --drive file=nbd://192.0.2.1:3 >>> @end example >>> >>> Example for Unix Domain Sockets >>> @example >>> -qemu-system-i386 --drive file=nbd:unix:/tmp/nbd-socket >>> +qemu-system-i386 --drive "file=nbd+unix:///?socket=/tmp/nbd-socket" >>> @end example >>> >>> @item SSH >>> >> >> Reviewed-by: John Snow > > Thanks; will queue through my NBD tree (regardless of whether we decide > I should add more patches to start a deprecation cycle). > -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH 1/1] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
---
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c..d60246d5d6 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
+
+"autoload" parameter is now ignored. All bitmaps are automatically loaded
+from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd641f1..e4975ece4a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1987,10 +1987,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -1999,7 +1995,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index fbef6845c8..93804da840 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH 0/1] dirty-bitmaps: remove deprecated autoload parameter
I'm going to be honest, here. There's actually no real reason to remove this now, but we could, so I'm going to. Also, in terms of the API serving as documentation, it's nicer to not pretend this is an option that does anything, so out it goes. This will serve as a little smoke test to see what happens if we actually stop dropping features we claimed were deprecated. John Snow (1): dirty-bitmaps: remove deprecated autoload parameter qemu-deprecated.texi | 20 +++- qapi/block-core.json | 6 +- blockdev.c | 6 -- 3 files changed, 16 insertions(+), 16 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/1] dirty-bitmaps: remove deprecated autoload parameter
On 9/25/19 3:20 AM, Vladimir Sementsov-Ogievskiy wrote: > 25.09.2019 2:01, John Snow wrote: >> This parameter has been deprecated since 2.12.0 and is eligible for >> removal. Remove this parameter as it is actually completely ignored; >> let's not give false hope. >> >> Signed-off-by: John Snow >> --- >> qemu-deprecated.texi | 20 +++- >> qapi/block-core.json | 6 +- >> blockdev.c | 6 -- >> 3 files changed, 16 insertions(+), 16 deletions(-) >> >> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi >> index 01245e0b1c..d60246d5d6 100644 >> --- a/qemu-deprecated.texi >> +++ b/qemu-deprecated.texi >> @@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of >> these three: >> >> @section QEMU Machine Protocol (QMP) commands >> >> -@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0) >> - >> -"autoload" parameter is now ignored. All bitmaps are automatically loaded >> -from qcow2 images. >> - >> @subsection query-block result field dirty-bitmaps[i].status (since 4.0) >> >> The ``status'' field of the ``BlockDirtyInfo'' structure, returned by >> @@ -356,3 +351,18 @@ existing CPU models. Management software that needs >> runnability >> guarantees must resolve the CPU model aliases using te >> ``alias-of'' field returned by the ``query-cpu-definitions'' QMP >> command. >> + >> + >> +@node Recently removed features >> +@appendix Recently removed features >> + >> +What follows is a record of recently removed, formerly deprecated >> +features that serves as a record for users who have encountered >> +trouble after a recent upgrade. >> + >> +@section QEMU Machine Protocol (QMP) commands >> + >> +@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0) > > Agree with Eric that it should be 4.2 - as this section is about removing > Yes, shame on me. I spent about three seconds on this patch and should have spent four. >> + >> +"autoload" parameter is now ignored. All bitmaps are automatically loaded >> +from qcow2 images. > > Maybe, rephrase it as s/is now ignored/is now removed (ignored since 2.12.0)/ > , > so that this paragraph don't mislead without a context. > Also a good idea. 'The "autoload" parameter has been ignored since 2.12.0. All bitmaps are automatically loaded from qcow2 images.' > Reviewed-by: Vladimir Sementsov-Ogievskiy > > (Yay, deprecation works!) > Thanks, and I'll get to the rest of your pending bitmap patches and cleanups soon. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] QEMU bitmap backup usability FAQ
On 8/20/19 6:25 PM, John Snow wrote: > Hi, downstream here at Red Hat I've been fielding some questions about > the usability and feature readiness of Bitmaps (and related features) in > QEMU. > > Here are some questions I answered internally that I am copying to the > list for two reasons: > > (1) To make sure my answers are actually correct, and > (2) To share this pseudo-reference with the community at large. > > This is long, and mostly for reference. There's a summary at the bottom > with some todo items and observations about the usability of the feature > as it exists in QEMU. > > Before too long, I intend to send a more summarized "roadmap" mail which > details all of the current and remaining work to be done in and around > the bitmaps feature in QEMU. > > > Questions: > >> "What format(s) is/are required for this functionality?" > > From the QEMU API, any format can be used to create and author > incremental backups. The only known format limitations are: > > 1. Persistent bitmaps cannot be created on any format except qcow2, > although there are hooks to add support to other formats at a later date > if desired. > > DANGER CAVEAT #1: Adding bitmaps to QEMU by default creates transient > bitmaps instead of persistent ones. > > Possible TODO: Allow users to 'upgrade' transient bitmaps to persistent > ones in case they made a mistake. > > > 2. When using push backups (blockdev-backup, drive-backup), you may use > any format as a target format. > > DANGER CAVEAT #2: without backing file and/or filesystem-less sparse > support, these images will be unusable. > > EXAMPLE: Backing up to a raw file loses allocation information, so we > can no longer distinguish between zeroes and unallocated regions. The > cluster size is also lost. This file will not be usable without > additional metadata recorded elsewhere.* > > (* This is complicated, but it is in theory possible to do a push backup > to e.g. an NBD target with custom server code that saves allocation > information to a metadata file, which would allow you to reconstruct > backups. For instance, recording in a .json file which extents were > written out would allow you to -- with a custom binary -- write this > information on top of a base file to reconstruct a backup.) > > > 3. Any format can be used for either shared storage or live storage > migrations. There are TWO distinct mechanisms for migrating bitmaps: > > A) The bitmap is flushed to storage and re-opened on the destination. > This is only supported for qcow2 and shared-storage migrations. > > B) The bitmap is live-migrated to the destination. This is supported for > any format and can be used for either shared storage or live storage > migrations. > > DANGER CAVEAT #3: The second bitmap migration technique there is an > optional migration capability that must be enabled explicitly. > Otherwise, some migration combinations may drop bitmaps. > > Matrix: > >> migrate = migrate_capability or (persistent and shared_storage) > > Enumerated: > > live storage + raw : transient + no-capability: Dropped > live-storage + raw : transient + bm-capability: Migrated > live-storage + qcow2 : transient + no-capability: Dropped > live-storage + qcow2 : transient + bm-capability: Migrated > live-storage + qcow2 : persistent + no-capability: Dropped (!) > live-storage + qcow2 : persistent + bm-capability: Migrated > > shared-storage + raw : transient - no-capability: Dropped > shared-storage + raw : transient + bm-capability: Migrated > shared-storage + qcow2 : transient + no-capability: Migrated > shared-storage + qcow2 : transient + bm-capability: Migrated > shared-storage + qcow2 : persistent + no-capability: Migrated > shared-storage + qcow2 : persistent + bm-capability: Migrated > > Enabling the bitmap migration capability will ALWAYS migrate the bitmap. > If it's disabled, we will only migrate the bitmaps for shared storage > migrations where the bitmap is persistent, which is a qcow2-only case. > > There is no warning or error if you attempt to migrate in a manner that > loses your bitmaps. > > (I might be persuaded to add a case for when you are doing a live > storage migration of qcow2 with persistent bitmaps, which is somewhat a > conflicting case: you've asked for the bitmap to be persistent, but it > seems likely that if this ever happens in practice, it's because you > have neglected to ask for it to be migrated to the new host.) > > See iotest 169 for more details on this. > > At present, these are the only format limitations I am consciously aware > of. From a managemen
Re: [libvirt] QEMU bitmap backup usability FAQ
On 9/25/19 11:11 AM, Vladimir Sementsov-Ogievskiy wrote: > 25.09.2019 16:52, John Snow wrote: >> >> >> On 8/20/19 6:25 PM, John Snow wrote: >>> Hi, downstream here at Red Hat I've been fielding some questions about >>> the usability and feature readiness of Bitmaps (and related features) in >>> QEMU. >>> >>> Here are some questions I answered internally that I am copying to the >>> list for two reasons: >>> >>> (1) To make sure my answers are actually correct, and >>> (2) To share this pseudo-reference with the community at large. >>> >>> This is long, and mostly for reference. There's a summary at the bottom >>> with some todo items and observations about the usability of the feature >>> as it exists in QEMU. >>> >>> Before too long, I intend to send a more summarized "roadmap" mail which >>> details all of the current and remaining work to be done in and around >>> the bitmaps feature in QEMU. >>> >>> >>> Questions: >>> >>>> "What format(s) is/are required for this functionality?" >>> >>> From the QEMU API, any format can be used to create and author >>> incremental backups. The only known format limitations are: >>> >>> 1. Persistent bitmaps cannot be created on any format except qcow2, >>> although there are hooks to add support to other formats at a later date >>> if desired. >>> >>> DANGER CAVEAT #1: Adding bitmaps to QEMU by default creates transient >>> bitmaps instead of persistent ones. >>> >>> Possible TODO: Allow users to 'upgrade' transient bitmaps to persistent >>> ones in case they made a mistake. >>> >>> >>> 2. When using push backups (blockdev-backup, drive-backup), you may use >>> any format as a target format. >>> >>> DANGER CAVEAT #2: without backing file and/or filesystem-less sparse >>> support, these images will be unusable. >>> >>> EXAMPLE: Backing up to a raw file loses allocation information, so we >>> can no longer distinguish between zeroes and unallocated regions. The >>> cluster size is also lost. This file will not be usable without >>> additional metadata recorded elsewhere.* >>> >>> (* This is complicated, but it is in theory possible to do a push backup >>> to e.g. an NBD target with custom server code that saves allocation >>> information to a metadata file, which would allow you to reconstruct >>> backups. For instance, recording in a .json file which extents were >>> written out would allow you to -- with a custom binary -- write this >>> information on top of a base file to reconstruct a backup.) >>> >>> >>> 3. Any format can be used for either shared storage or live storage >>> migrations. There are TWO distinct mechanisms for migrating bitmaps: >>> >>> A) The bitmap is flushed to storage and re-opened on the destination. >>> This is only supported for qcow2 and shared-storage migrations. >>> >>> B) The bitmap is live-migrated to the destination. This is supported for >>> any format and can be used for either shared storage or live storage >>> migrations. >>> >>> DANGER CAVEAT #3: The second bitmap migration technique there is an >>> optional migration capability that must be enabled explicitly. >>> Otherwise, some migration combinations may drop bitmaps. >>> >>> Matrix: >>> >>>> migrate = migrate_capability or (persistent and shared_storage) >>> >>> Enumerated: >>> >>> live storage + raw : transient + no-capability: Dropped >>> live-storage + raw : transient + bm-capability: Migrated >>> live-storage + qcow2 : transient + no-capability: Dropped >>> live-storage + qcow2 : transient + bm-capability: Migrated >>> live-storage + qcow2 : persistent + no-capability: Dropped (!) >>> live-storage + qcow2 : persistent + bm-capability: Migrated >>> >>> shared-storage + raw : transient - no-capability: Dropped >>> shared-storage + raw : transient + bm-capability: Migrated >>> shared-storage + qcow2 : transient + no-capability: Migrated >>> shared-storage + qcow2 : transient + bm-capability: Migrated >>> shared-storage + qcow2 : persistent + no-capability: Migrated >>> shared-storage + qcow2 : persistent + bm-capability: Migrated >>> >>> Enabling the bitmap migration capability will ALWAYS migrate the b
Re: [libvirt] [PATCH v2] dirty-bitmaps: remove deprecated autoload parameter
On 10/2/19 7:22 PM, John Snow wrote: > This parameter has been deprecated since 2.12.0 and is eligible for > removal. Remove this parameter as it is actually completely ignored; > let's not give false hope. > > Signed-off-by: John Snow > Reviewed-by: Eric Blake > Reviewed-by: Vladimir Sementsov-Ogievskiy bah, NACK; I didn't commit the changes I made. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v3] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
V2: Change 2.12.0 -> 4.2.0 in removed section.
Adjust phrasing to match.
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c..7239e0959d 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 4.2.0)
+
+The "autoload" parameter has been ignored since 2.12.0. All bitmaps
+are automatically loaded from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd641f1..e4975ece4a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1987,10 +1987,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -1999,7 +1995,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index fbef6845c8..93804da840 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
---
V2: Change 2.12.0 -> 4.2.0 in removed section.
Adjust phrasing to match.
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c..d60246d5d6 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
+
+"autoload" parameter is now ignored. All bitmaps are automatically loaded
+from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd641f1..e4975ece4a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1987,10 +1987,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -1999,7 +1995,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index fbef6845c8..93804da840 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH 1/1] IDE: deprecate ide-drive
It's an old compatibility shim that just delegates to ide-cd or ide-hd.
I'd like to refactor these some day, and getting rid of the super-object
will make that easier.
Either way, we don't need this.
Signed-off-by: John Snow
---
qemu-deprecated.texi | 5 +
hw/ide/qdev.c | 3 +++
tests/qemu-iotests/051.pc.out | 6 --
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c4..f802d83983e 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -247,6 +247,11 @@ quite a bit. It will be removed without replacement unless
some users speaks
up at the @email{qemu-devel@@nongnu.org} mailing list with information about
their usecases.
+@subsection ide-drive (since 4.2)
+
+The 'ide-drive' device is deprecated. Users should use 'ide-hd' or
+'ide-cd' as appropriate to get an IDE hard disk or CDROM as needed.
+
@section System emulator machines
@subsection pc-0.12, pc-0.13, pc-0.14 and pc-0.15 (since 4.0)
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 6fba6b62b87..9ecee4da074 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -279,6 +279,9 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
{
DriveInfo *dinfo = NULL;
+warn_report("The 'ide-drive' device is deprecated. "
+"Use 'ide-hd' or 'ide-cd' instead");
+
if (dev->conf.blk) {
dinfo = blk_legacy_dinfo(dev->conf.blk);
}
diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index 000557c7c83..93b9a1f82ca 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -158,7 +158,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive if=none,id=disk -device ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive
is empty
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: The 'ide-drive'
device is deprecated. Use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive is empty
Testing: -drive if=none,id=disk -device ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
@@ -228,7 +229,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: The 'ide-drive'
device is deprecated. Use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH 0/1] IDE: Deprecate ide-drive
John Snow (1): IDE: deprecate ide-drive qemu-deprecated.texi | 5 + hw/ide/qdev.c | 3 +++ tests/qemu-iotests/051.pc.out | 6 -- 3 files changed, 12 insertions(+), 2 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/1] IDE: deprecate ide-drive
On 10/7/19 5:49 AM, Markus Armbruster wrote:
> John Snow writes:
>
>> It's an old compatibility shim that just delegates to ide-cd or ide-hd.
>> I'd like to refactor these some day, and getting rid of the super-object
>> will make that easier.
>
> Device "scsi-disk" is similar. However, it's still used by the
> scsi_bus_legacy_add_drive() magic. Not sure that's fully deprecated,
> yet. If / once it is, we can deprecate "scsi-disk", too. Anyway, not
> your department.
>
Yeah. I just want to get rid of this to allow myself to do bolder things
later on.
I have literally no time to do this and it's not really anything that
would make anyone money, but...
I want to add a few explicit devices:
ata-hd
ata-cd
sata-hd
sata-cd
With some shared state structures that implement common feature subsets,
like ata_registers, sata_registers, atapi_registers, etc.
I'd also like to separate out frontend and backend state providing a bit
of a cleaner division between device configuration (parameters on the
hardware creation itself), emulated device state (ATA register sets and
state machine), and QEMU backend state (block_backend pointers, aio
state counters, locks, etc etc etc -- Things solely purposed for
interacting with the block module.)
I'd also like to make each device type plug into ATA or SATA bus slots
explicitly -- no more magic IDE devices.
It's like the 5-year itch I can't help but want to scratch. My name's on
this code and it's UGLY UGLY UGLY!
The biggest roadblock to me actually doing this is figuring out how it
would be even vaguely possible to migrate from ide-hd or ide-cd to the
newer models -- it might be pretty complex, but maybe I can figure
something out somehow...
Well, suggestions welcome.
>> Either way, we don't need this.
>>
>> Signed-off-by: John Snow
>> ---
>> qemu-deprecated.texi | 5 +
>> hw/ide/qdev.c | 3 +++
>> tests/qemu-iotests/051.pc.out | 6 --
>> 3 files changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
>> index 01245e0b1c4..f802d83983e 100644
>> --- a/qemu-deprecated.texi
>> +++ b/qemu-deprecated.texi
>> @@ -247,6 +247,11 @@ quite a bit. It will be removed without replacement
>> unless some users speaks
>> up at the @email{qemu-devel@@nongnu.org} mailing list with information about
>> their usecases.
>>
>> +@subsection ide-drive (since 4.2)
>> +
>> +The 'ide-drive' device is deprecated. Users should use 'ide-hd' or
>> +'ide-cd' as appropriate to get an IDE hard disk or CDROM as needed.
>
> CD-ROM
>
>:[
>> +
>> @section System emulator machines
>>
>> @subsection pc-0.12, pc-0.13, pc-0.14 and pc-0.15 (since 4.0)
>> diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
>> index 6fba6b62b87..9ecee4da074 100644
>> --- a/hw/ide/qdev.c
>> +++ b/hw/ide/qdev.c
>> @@ -279,6 +279,9 @@ static void ide_drive_realize(IDEDevice *dev, Error
>> **errp)
>> {
>> DriveInfo *dinfo = NULL;
>>
>> +warn_report("The 'ide-drive' device is deprecated. "
>> +"Use 'ide-hd' or 'ide-cd' instead");
>
> Two sentences, where only the first one terminated with a period.
>
> Let's say "is deprecated, please use", like we do in several other places.
>
Alright.
>> +
>> if (dev->conf.blk) {
>> dinfo = blk_legacy_dinfo(dev->conf.blk);
>> }
>> diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
>> index 000557c7c83..93b9a1f82ca 100644
>> --- a/tests/qemu-iotests/051.pc.out
>> +++ b/tests/qemu-iotests/051.pc.out
>> @@ -158,7 +158,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
>>
>> Testing: -drive if=none,id=disk -device ide-drive,drive=disk
>> QEMU X.Y.Z monitor - type 'help' for more information
>> -(qemu) QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but
>> drive is empty
>> +(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: The 'ide-drive'
>> device is deprecated. Use 'ide-hd' or 'ide-cd' instead
>> +QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive is
>> empty
>>
>> Testing: -drive if=none,id=disk -device ide-hd,drive=disk
>> QEMU X.Y.Z monitor - type 'help' for more information
>> @@ -228,7 +229,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
>>
>> Testing: -drive file=TEST_DIR/t.q
Re: [libvirt] [PATCH v3] dirty-bitmaps: remove deprecated autoload parameter
On 10/2/19 7:24 PM, John Snow wrote: > This parameter has been deprecated since 2.12.0 and is eligible for > removal. Remove this parameter as it is actually completely ignored; > let's not give false hope. > > Signed-off-by: John Snow > Reviewed-by: Eric Blake > Reviewed-by: Vladimir Sementsov-Ogievskiy Thanks, applied to my bitmaps tree: https://github.com/jnsnow/qemu/commits/bitmaps https://github.com/jnsnow/qemu.git --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2 0/1] IDE: Deprecate ide-drive
V2: Change phrasings and spellings as Markus suggested. John Snow (1): IDE: deprecate ide-drive qemu-deprecated.texi | 5 + hw/ide/qdev.c | 3 +++ tests/qemu-iotests/051.pc.out | 6 -- 3 files changed, 12 insertions(+), 2 deletions(-) -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH 1/1] IDE: deprecate ide-drive
On 10/8/19 2:51 AM, Markus Armbruster wrote: >> I'll respin to hit the tests with a stiffer scrub-brush. > Thanks! 051 is the only test I can find that uses ide-drive, and the non-pc version of the test doesn't seem to use it, so this actually seems sufficient. I'd like to keep the test for ide-drive itself in until we actually remove it, just to make sure it still works. No harm in keeping it, I think. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PATCH v2 1/1] IDE: deprecate ide-drive
It's an old compatibility shim that just delegates to ide-cd or ide-hd.
I'd like to refactor these some day, and getting rid of the super-object
will make that easier.
Either way, we don't need this.
Libvirt-checked-by: Peter Krempa
Signed-off-by: John Snow
---
qemu-deprecated.texi | 5 +
hw/ide/qdev.c | 3 +++
tests/qemu-iotests/051.pc.out | 6 --
3 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c4..7cd4648df3c 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -247,6 +247,11 @@ quite a bit. It will be removed without replacement unless
some users speaks
up at the @email{qemu-devel@@nongnu.org} mailing list with information about
their usecases.
+@subsection ide-drive (since 4.2)
+
+The 'ide-drive' device is deprecated. Users should use 'ide-hd' or
+'ide-cd' as appropriate to get an IDE hard disk or CD-ROM as needed.
+
@section System emulator machines
@subsection pc-0.12, pc-0.13, pc-0.14 and pc-0.15 (since 4.0)
diff --git a/hw/ide/qdev.c b/hw/ide/qdev.c
index 6fba6b62b87..3666e597211 100644
--- a/hw/ide/qdev.c
+++ b/hw/ide/qdev.c
@@ -279,6 +279,9 @@ static void ide_drive_realize(IDEDevice *dev, Error **errp)
{
DriveInfo *dinfo = NULL;
+warn_report("'ide-drive' is deprecated, "
+"please use 'ide-hd' or 'ide-cd' instead");
+
if (dev->conf.blk) {
dinfo = blk_legacy_dinfo(dev->conf.blk);
}
diff --git a/tests/qemu-iotests/051.pc.out b/tests/qemu-iotests/051.pc.out
index 000557c7c83..34849dd1720 100644
--- a/tests/qemu-iotests/051.pc.out
+++ b/tests/qemu-iotests/051.pc.out
@@ -158,7 +158,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive if=none,id=disk -device ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive
is empty
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Device needs media, but drive is empty
Testing: -drive if=none,id=disk -device ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
@@ -228,7 +229,8 @@ QEMU X.Y.Z monitor - type 'help' for more information
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-drive,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
-(qemu) QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
+(qemu) QEMU_PROG: -device ide-drive,drive=disk: warning: 'ide-drive' is
deprecated, please use 'ide-hd' or 'ide-cd' instead
+QEMU_PROG: -device ide-drive,drive=disk: Block node is read-only
Testing: -drive file=TEST_DIR/t.qcow2,if=none,id=disk,readonly=on -device
ide-hd,drive=disk
QEMU X.Y.Z monitor - type 'help' for more information
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 1/1] IDE: deprecate ide-drive
On 10/10/19 7:54 AM, Peter Krempa wrote: > On Thu, Oct 10, 2019 at 13:42:26 +0200, Philippe Mathieu-Daudé wrote: >> On 10/10/19 1:26 PM, Peter Krempa wrote: >>> On Thu, Oct 10, 2019 at 13:22:37 +0200, Philippe Mathieu-Daudé wrote: >>>> On 10/10/19 12:43 AM, John Snow wrote: >>>>> It's an old compatibility shim that just delegates to ide-cd or ide-hd. >>>>> I'd like to refactor these some day, and getting rid of the super-object >>>>> will make that easier. >>>>> >>>>> Either way, we don't need this. >>>>> >>>>> Libvirt-checked-by: Peter Krempa >>>> >>>> Peter made a comment regarding Laszlo's Regression-tested-by tag: >>>> >>>>[...] nobody else is using >>>>this convention (there are exactly 0 instances of >>>>"Regression-tested-by" in the project git log as far as >>>>I can see), and so in practice people reading the commits >>>>won't really know what you meant by it. Everybody else >>>>on the project uses "Tested-by" to mean either of the >>>>two cases you describe above, without distinction... >>>> >>>> It probably applies to 'Libvirt-checked-by' too. >>> >>> I certainly didn't test it. So feel free to drop that line altogether. >> >> But you reviewed it, can we use your 'Reviewed-by' instead? > > To be honest, I didn't really review the code nor the documentation. > I actually reviewed only the idea itself in the context of integration > with libvirt and that's why I didn't go for 'Reviewed-by:'. > > The gist of the citation above is that we should stick to well known > tags with their well known meanings and I think that considering this a > 'review' would be a stretch of the definiton. > I wasn't aware that PMM wanted to avoid non-standard tags; I consider them to be for human use, but I can change that behavior. Peter, I'll change it to an ACK (as suggested by Kevin) is that's OK by you. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 1/1] IDE: deprecate ide-drive
On 10/11/19 5:12 AM, Peter Krempa wrote: > On Thu, Oct 10, 2019 at 14:08:12 -0400, John Snow wrote: >> On 10/10/19 7:54 AM, Peter Krempa wrote: >>> On Thu, Oct 10, 2019 at 13:42:26 +0200, Philippe Mathieu-Daudé wrote: >>>> On 10/10/19 1:26 PM, Peter Krempa wrote: > > [...] > >>> To be honest, I didn't really review the code nor the documentation. >>> I actually reviewed only the idea itself in the context of integration >>> with libvirt and that's why I didn't go for 'Reviewed-by:'. >>> >>> The gist of the citation above is that we should stick to well known >>> tags with their well known meanings and I think that considering this a >>> 'review' would be a stretch of the definiton. >>> >> >> I wasn't aware that PMM wanted to avoid non-standard tags; I consider >> them to be for human use, but I can change that behavior. >> >> Peter, I'll change it to an ACK (as suggested by Kevin) is that's OK by you. > > Sure! I'll spell it out even for compliance: > > ACKed-by: Peter Krempa > Thanks, applied to my IDE tree: https://github.com/jnsnow/qemu/commits/ide https://github.com/jnsnow/qemu.git --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 03/19] block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap
From: Vladimir Sementsov-Ogievskiy
It's more comfortable to not deal with local_err.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190920082543.23444-3-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 5 ++---
include/block/block_int.h| 6 +++---
include/block/dirty-bitmap.h | 5 ++---
block/dirty-bitmap.c | 9 +
block/qcow2-bitmap.c | 18 ++
blockdev.c | 7 +++
6 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index a488d761ff0..2ed54821635 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -747,9 +747,8 @@ bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
const char *name,
uint32_t granularity,
Error **errp);
-void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp);
+int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char
*name,
+ Error **errp);
ssize_t coroutine_fn
qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 0422acdf1c4..503ac9e3cd2 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -556,9 +556,9 @@ struct BlockDriver {
const char *name,
uint32_t granularity,
Error **errp);
-void (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
-const char *name,
-Error **errp);
+int (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+ const char *name,
+ Error **errp);
/**
* Register/unregister a buffer for I/O. For example, when the driver is
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 4b4b731b469..07503b03b53 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -37,9 +37,8 @@ int bdrv_dirty_bitmap_check(const BdrvDirtyBitmap *bitmap,
uint32_t flags,
Error **errp);
void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs);
-void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp);
+int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+Error **errp);
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 8f42015db95..d1ae2e19229 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -455,13 +455,14 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState
*bs)
* not fail.
* This function doesn't release corresponding BdrvDirtyBitmap.
*/
-void bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp)
+int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+Error **errp)
{
if (bs->drv && bs->drv->bdrv_remove_persistent_dirty_bitmap) {
-bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp);
+return bs->drv->bdrv_remove_persistent_dirty_bitmap(bs, name, errp);
}
+
+return 0;
}
bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index b2487101ede..9821c1628f5 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1404,9 +1404,8 @@ static Qcow2Bitmap *find_bitmap_by_name(Qcow2BitmapList
*bm_list,
return NULL;
}
-void qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- Error **errp)
+int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char
*name,
+ Error **errp)
{
int ret;
BDRVQcow2State *s = bs->opaque;
@@ -1416,18 +1415,19 @@ void
qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs,
if (s->nb_bitmaps == 0) {
/* Absence of the
[libvirt] [PULL 04/19] block/qcow2: proper locking on bitmap add/remove paths
From: Vladimir Sementsov-Ogievskiy
qmp_block_dirty_bitmap_add and do_block_dirty_bitmap_remove do acquire
aio context since 0a6c86d024c52b. But this is not enough: we also must
lock qcow2 mutex when access in-image metadata. Especially it concerns
freeing qcow2 clusters.
To achieve this, move qcow2_can_store_new_dirty_bitmap and
qcow2_remove_persistent_dirty_bitmap to coroutine context.
Since we work in coroutines in correct aio context, we don't need
context acquiring in blockdev.c anymore, drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190920082543.23444-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h | 11 ++--
include/block/block_int.h | 10 ++--
block/dirty-bitmap.c | 102 +++---
block/qcow2-bitmap.c | 24 ++---
block/qcow2.c | 5 +-
blockdev.c| 27 +++---
6 files changed, 131 insertions(+), 48 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 2ed54821635..113d99bf520 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -743,12 +743,13 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
-bool qcow2_can_store_new_dirty_bitmap(BlockDriverState *bs,
- const char *name,
- uint32_t granularity,
- Error **errp);
-int qcow2_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char
*name,
+bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
+ const char *name,
+ uint32_t granularity,
Error **errp);
+int qcow2_co_remove_persistent_dirty_bitmap(BlockDriverState *bs,
+const char *name,
+Error **errp);
ssize_t coroutine_fn
qcow2_co_compress(BlockDriverState *bs, void *dest, size_t dest_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 503ac9e3cd2..1e54486ad14 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -552,13 +552,13 @@ struct BlockDriver {
* field of BlockDirtyBitmap's in case of success.
*/
int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
-bool (*bdrv_can_store_new_dirty_bitmap)(BlockDriverState *bs,
-const char *name,
-uint32_t granularity,
-Error **errp);
-int (*bdrv_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+bool (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,
const char *name,
+ uint32_t granularity,
Error **errp);
+int (*bdrv_co_remove_persistent_dirty_bitmap)(BlockDriverState *bs,
+ const char *name,
+ Error **errp);
/**
* Register/unregister a buffer for I/O. For example, when the driver is
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index d1ae2e19229..03e0872b972 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -26,6 +26,7 @@
#include "trace.h"
#include "block/block_int.h"
#include "block/blockjob.h"
+#include "qemu/main-loop.h"
struct BdrvDirtyBitmap {
QemuMutex *mutex;
@@ -455,18 +456,59 @@ void bdrv_release_named_dirty_bitmaps(BlockDriverState
*bs)
* not fail.
* This function doesn't release corresponding BdrvDirtyBitmap.
*/
+static int coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
+ Error **errp)
+{
+if (bs->drv && bs->drv->bdrv_co_remove_persistent_dirty_bitmap) {
+return bs->drv->bdrv_co_remove_persistent_dirty_bitmap(bs, name, errp);
+}
+
+return 0;
+}
+
+typedef struct BdrvRemovePersistentDirtyBitmapCo {
+BlockDriverState *bs;
+const char *name;
+Error **errp;
+int ret;
+} BdrvRemovePersistentDirtyBitmapCo;
+
+static void coroutine_fn
+bdrv_co_remove_persistent_dirty_bitmap_entry(void *opaque)
+{
+BdrvRemovePersistentDirtyBitmapCo *s = opaque;
+
+s->ret = bdrv_co_remove_persistent_dirty_bitmap(s->bs, s->name, s->errp);
+aio_wait_kick();
+}
+
int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
Error **errp)
[libvirt] [PULL 01/19] util/hbitmap: strict hbitmap_reset
From: Vladimir Sementsov-Ogievskiy hbitmap_reset has an unobvious property: it rounds requested region up. It may provoke bugs, like in recently fixed write-blocking mode of mirror: user calls reset on unaligned region, not keeping in mind that there are possible unrelated dirty bytes, covered by rounded-up region and information of this unrelated "dirtiness" will be lost. Make hbitmap_reset strict: assert that arguments are aligned, allowing only one exception when @start + @count == hb->orig_size. It's needed to comfort users of hbitmap_next_dirty_area, which cares about hb->orig_size. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Message-Id: <20190806152611.280389-1-vsement...@virtuozzo.com> [Maintainer edit: Max's suggestions from on-list. --js] Signed-off-by: John Snow --- include/qemu/hbitmap.h | 5 + tests/test-hbitmap.c | 2 +- util/hbitmap.c | 4 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h index 4afbe6292e3..1bf944ca3d1 100644 --- a/include/qemu/hbitmap.h +++ b/include/qemu/hbitmap.h @@ -132,6 +132,11 @@ void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count); * @count: Number of bits to reset. * * Reset a consecutive range of bits in an HBitmap. + * @start and @count must be aligned to bitmap granularity. The only exception + * is resetting the tail of the bitmap: @count may be equal to hb->orig_size - + * @start, in this case @count may be not aligned. The sum of @start + @count is + * allowed to be greater than hb->orig_size, but only if @start < hb->orig_size + * and @start + @count = ALIGN_UP(hb->orig_size, granularity). */ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count); diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c index eed5d288cbc..e1f867085f4 100644 --- a/tests/test-hbitmap.c +++ b/tests/test-hbitmap.c @@ -423,7 +423,7 @@ static void test_hbitmap_granularity(TestHBitmapData *data, hbitmap_test_check(data, 0); hbitmap_test_set(data, 0, 3); g_assert_cmpint(hbitmap_count(data->hb), ==, 4); -hbitmap_test_reset(data, 0, 1); +hbitmap_test_reset(data, 0, 2); g_assert_cmpint(hbitmap_count(data->hb), ==, 2); } diff --git a/util/hbitmap.c b/util/hbitmap.c index fd44c897ab0..757d39e360a 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -476,6 +476,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count) /* Compute range in the last layer. */ uint64_t first; uint64_t last = start + count - 1; +uint64_t gran = 1ULL << hb->granularity; + +assert(!(start & (gran - 1))); +assert(!(count & (gran - 1)) || (start + count == hb->orig_size)); trace_hbitmap_reset(hb, start, count, start >> hb->granularity, last >> hb->granularity); -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 00/19] Bitmaps patches
The following changes since commit 98b2e3c9ab3abfe476a2b02f8f51813edb90e72d: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2019-10-08 16:08:35 +0100) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull-request for you to fetch changes up to b97d9a1014b61dd0980e7f4a0c9ca1e3b0aaa761: dirty-bitmaps: remove deprecated autoload parameter (2019-10-09 17:02:45 -0400) Pull request ---- John Snow (2): MAINTAINERS: Add Vladimir as a reviewer for bitmaps dirty-bitmaps: remove deprecated autoload parameter Vladimir Sementsov-Ogievskiy (17): util/hbitmap: strict hbitmap_reset block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c block/dirty-bitmap: return int from bdrv_remove_persistent_dirty_bitmap block/qcow2: proper locking on bitmap add/remove paths block/dirty-bitmap: drop meta block/dirty-bitmap: add bs link block/dirty-bitmap: drop BdrvDirtyBitmap.mutex block/dirty-bitmap: refactor bdrv_dirty_bitmap_next block: switch reopen queue from QSIMPLEQ to QTAILQ block: reverse order for reopen commits iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint() block/qcow2-bitmap: do not remove bitmaps on reopen-ro iotests: add test 260 to check bitmap life after snapshot + commit block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit qemu-deprecated.texi | 20 ++- qapi/block-core.json | 6 +- block/qcow2.h | 19 +-- include/block/block.h | 2 +- include/block/block_int.h | 20 +-- include/block/dirty-bitmap.h | 34 ++-- include/qemu/hbitmap.h | 5 + block.c| 79 +++-- block/backup.c | 14 +- block/dirty-bitmap.c | 290 +++-- block/mirror.c | 4 +- block/qcow2-bitmap.c | 212 +++- block/qcow2.c | 22 ++- blockdev.c | 40 ++--- migration/block-dirty-bitmap.c | 11 +- migration/block.c | 4 +- tests/test-hbitmap.c | 2 +- util/hbitmap.c | 4 + MAINTAINERS| 3 +- tests/qemu-iotests/165 | 57 ++- tests/qemu-iotests/165.out | 4 +- tests/qemu-iotests/260 | 89 ++ tests/qemu-iotests/260.out | 52 ++ tests/qemu-iotests/group | 1 + 24 files changed, 624 insertions(+), 370 deletions(-) create mode 100755 tests/qemu-iotests/260 create mode 100644 tests/qemu-iotests/260.out -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 02/19] block: move bdrv_can_store_new_dirty_bitmap to block/dirty-bitmap.c
From: Vladimir Sementsov-Ogievskiy
block/dirty-bitmap.c seems to be more appropriate for it and
bdrv_remove_persistent_dirty_bitmap already in it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190920082543.23444-2-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block.c | 22 --
block/dirty-bitmap.c | 22 ++
2 files changed, 22 insertions(+), 22 deletions(-)
diff --git a/block.c b/block.c
index 59441248451..bea03cfcc92 100644
--- a/block.c
+++ b/block.c
@@ -6555,25 +6555,3 @@ void bdrv_del_child(BlockDriverState *parent_bs,
BdrvChild *child, Error **errp)
parent_bs->drv->bdrv_del_child(parent_bs, child, errp);
}
-
-bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
- uint32_t granularity, Error **errp)
-{
-BlockDriver *drv = bs->drv;
-
-if (!drv) {
-error_setg_errno(errp, ENOMEDIUM,
- "Can't store persistent bitmaps to %s",
- bdrv_get_device_or_node_name(bs));
-return false;
-}
-
-if (!drv->bdrv_can_store_new_dirty_bitmap) {
-error_setg_errno(errp, ENOTSUP,
- "Can't store persistent bitmaps to %s",
- bdrv_get_device_or_node_name(bs));
-return false;
-}
-
-return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp);
-}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 134e0c9a0c8..8f42015db95 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -464,6 +464,28 @@ void bdrv_remove_persistent_dirty_bitmap(BlockDriverState
*bs,
}
}
+bool bdrv_can_store_new_dirty_bitmap(BlockDriverState *bs, const char *name,
+ uint32_t granularity, Error **errp)
+{
+BlockDriver *drv = bs->drv;
+
+if (!drv) {
+error_setg_errno(errp, ENOMEDIUM,
+ "Can't store persistent bitmaps to %s",
+ bdrv_get_device_or_node_name(bs));
+return false;
+}
+
+if (!drv->bdrv_can_store_new_dirty_bitmap) {
+error_setg_errno(errp, ENOTSUP,
+ "Can't store persistent bitmaps to %s",
+ bdrv_get_device_or_node_name(bs));
+return false;
+}
+
+return drv->bdrv_can_store_new_dirty_bitmap(bs, name, granularity, errp);
+}
+
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
bdrv_dirty_bitmap_lock(bitmap);
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 07/19] block/dirty-bitmap: drop BdrvDirtyBitmap.mutex
From: Vladimir Sementsov-Ogievskiy
mutex field is just a pointer to bs->dirty_bitmap_mutex, so no needs
to store it in BdrvDirtyBitmap when we have bs pointer in it (since
previous patch).
Drop mutex field. Constantly use bdrv_dirty_bitmaps_lock/unlock in
block/dirty-bitmap.c to make it more obvious that it's not per-bitmap
lock. Still, for simplicity, leave bdrv_dirty_bitmap_lock/unlock
functions as an external API.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/dirty-bitmap.c | 84 +---
1 file changed, 41 insertions(+), 43 deletions(-)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 44453ff8241..4e5c87a907f 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -29,7 +29,6 @@
#include "qemu/main-loop.h"
struct BdrvDirtyBitmap {
-QemuMutex *mutex;
BlockDriverState *bs;
HBitmap *bitmap;/* Dirty bitmap implementation */
bool busy; /* Bitmap is busy, it can't be used via QMP */
@@ -72,12 +71,12 @@ static inline void
bdrv_dirty_bitmaps_unlock(BlockDriverState *bs)
void bdrv_dirty_bitmap_lock(BdrvDirtyBitmap *bitmap)
{
-qemu_mutex_lock(bitmap->mutex);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
}
void bdrv_dirty_bitmap_unlock(BdrvDirtyBitmap *bitmap)
{
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called with BQL or dirty_bitmap lock taken. */
@@ -117,7 +116,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
}
bitmap = g_new0(BdrvDirtyBitmap, 1);
bitmap->bs = bs;
-bitmap->mutex = &bs->dirty_bitmap_mutex;
bitmap->bitmap = hbitmap_alloc(bitmap_size, ctz32(granularity));
bitmap->size = bitmap_size;
bitmap->name = g_strdup(name);
@@ -151,9 +149,9 @@ static bool bdrv_dirty_bitmap_busy(const BdrvDirtyBitmap
*bitmap)
void bdrv_dirty_bitmap_set_busy(BdrvDirtyBitmap *bitmap, bool busy)
{
-qemu_mutex_lock(bitmap->mutex);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bitmap->busy = busy;
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called with BQL taken. */
@@ -278,10 +276,10 @@ void bdrv_enable_dirty_bitmap_locked(BdrvDirtyBitmap
*bitmap)
/* Called with BQL taken. */
void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap)
{
-assert(bitmap->mutex == bitmap->successor->mutex);
-qemu_mutex_lock(bitmap->mutex);
+assert(bitmap->bs == bitmap->successor->bs);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_enable_dirty_bitmap_locked(bitmap->successor);
-qemu_mutex_unlock(bitmap->mutex);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called within bdrv_dirty_bitmap_lock..unlock and with BQL taken. */
@@ -361,9 +359,9 @@ BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BdrvDirtyBitmap
*parent,
{
BdrvDirtyBitmap *ret;
-qemu_mutex_lock(parent->mutex);
+bdrv_dirty_bitmaps_lock(parent->bs);
ret = bdrv_reclaim_dirty_bitmap_locked(parent, errp);
-qemu_mutex_unlock(parent->mutex);
+bdrv_dirty_bitmaps_unlock(parent->bs);
return ret;
}
@@ -543,16 +541,16 @@ bool bdrv_can_store_new_dirty_bitmap(BlockDriverState
*bs, const char *name,
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bitmap->disabled = true;
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_enable_dirty_bitmap_locked(bitmap);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs)
@@ -593,9 +591,9 @@ bool bdrv_dirty_bitmap_get_locked(BdrvDirtyBitmap *bitmap,
int64_t offset)
bool bdrv_dirty_bitmap_get(BdrvDirtyBitmap *bitmap, int64_t offset)
{
bool ret;
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
ret = bdrv_dirty_bitmap_get_locked(bitmap, offset);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
return ret;
}
@@ -660,9 +658,9 @@ void bdrv_set_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes)
{
-bdrv_dirty_bitmap_lock(bitmap);
+bdrv_dirty_bitmaps_lock(bitmap->bs);
bdrv_set_dirty_bitmap_locked(bitmap, offset, bytes);
-bdrv_dirty_bitmap_unlock(bitmap);
+bdrv_dirty_bitmaps_unlock(bitmap->bs);
}
/* Called within bdrv_dirty_bitmap_lock..unl
[libvirt] [PULL 05/19] block/dirty-bitmap: drop meta
From: Vladimir Sementsov-Ogievskiy
Drop meta bitmaps, as they are unused.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-2-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 5
block/dirty-bitmap.c | 46
2 files changed, 51 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 07503b03b53..973056778aa 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -18,9 +18,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
uint32_t granularity,
const char *name,
Error **errp);
-void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
- int chunk_size);
-void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap);
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
Error **errp);
@@ -54,7 +51,6 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes);
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t offset, int64_t bytes);
-BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap);
BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter);
@@ -96,7 +92,6 @@ void bdrv_reset_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter);
void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
-int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_bitmap_truncate(BlockDriverState *bs, int64_t bytes);
bool bdrv_dirty_bitmap_readonly(const BdrvDirtyBitmap *bitmap);
bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 03e0872b972..4ecf18d5df7 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -31,7 +31,6 @@
struct BdrvDirtyBitmap {
QemuMutex *mutex;
HBitmap *bitmap;/* Dirty bitmap implementation */
-HBitmap *meta; /* Meta dirty bitmap */
bool busy; /* Bitmap is busy, it can't be used via QMP */
BdrvDirtyBitmap *successor; /* Anonymous child, if any. */
char *name; /* Optional non-empty unique ID */
@@ -127,36 +126,6 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
return bitmap;
}
-/* bdrv_create_meta_dirty_bitmap
- *
- * Create a meta dirty bitmap that tracks the changes of bits in @bitmap. I.e.
- * when a dirty status bit in @bitmap is changed (either from reset to set or
- * the other way around), its respective meta dirty bitmap bit will be marked
- * dirty as well.
- *
- * @bitmap: the block dirty bitmap for which to create a meta dirty bitmap.
- * @chunk_size: how many bytes of bitmap data does each bit in the meta bitmap
- * track.
- */
-void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
- int chunk_size)
-{
-assert(!bitmap->meta);
-qemu_mutex_lock(bitmap->mutex);
-bitmap->meta = hbitmap_create_meta(bitmap->bitmap,
- chunk_size * BITS_PER_BYTE);
-qemu_mutex_unlock(bitmap->mutex);
-}
-
-void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap)
-{
-assert(bitmap->meta);
-qemu_mutex_lock(bitmap->mutex);
-hbitmap_free_meta(bitmap->bitmap);
-bitmap->meta = NULL;
-qemu_mutex_unlock(bitmap->mutex);
-}
-
int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap)
{
return bitmap->size;
@@ -320,7 +289,6 @@ static void
bdrv_release_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap)
assert(!bitmap->active_iterators);
assert(!bdrv_dirty_bitmap_busy(bitmap));
assert(!bdrv_dirty_bitmap_has_successor(bitmap));
-assert(!bitmap->meta);
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
g_free(bitmap->name);
@@ -666,15 +634,6 @@ BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap
*bitmap)
return iter;
}
-BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap)
-{
-BdrvDirtyBitmapIter *iter = g_new(BdrvDirtyBitmapIter, 1);
-hbitmap_iter_init(&iter->hbi, bitmap->meta, 0);
-iter->bitmap = bitmap;
-bitmap->active_iterators++;
-return iter;
-}
-
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter)
{
if (!iter) {
@@ -821,11 +780,6 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
return hbitmap_count(bitmap->bitmap);
}
-int64_t bdrv_get_m
[libvirt] [PULL 11/19] iotests: add test-case to 165 to test reopening qcow2 bitmaps to RW
From: Vladimir Sementsov-Ogievskiy
Reopening bitmaps to RW was broken prior to previous commit. Check that
it works now.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-4-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
tests/qemu-iotests/165 | 57 --
tests/qemu-iotests/165.out | 4 +--
2 files changed, 57 insertions(+), 4 deletions(-)
diff --git a/tests/qemu-iotests/165 b/tests/qemu-iotests/165
index 5650dc7c874..951ea011a27 100755
--- a/tests/qemu-iotests/165
+++ b/tests/qemu-iotests/165
@@ -43,10 +43,10 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
os.remove(disk)
def mkVm(self):
-return iotests.VM().add_drive(disk)
+return iotests.VM().add_drive(disk, opts='node-name=node0')
def mkVmRo(self):
-return iotests.VM().add_drive(disk, opts='readonly=on')
+return iotests.VM().add_drive(disk, opts='readonly=on,node-name=node0')
def getSha256(self):
result = self.vm.qmp('x-debug-block-dirty-bitmap-sha256',
@@ -102,6 +102,59 @@ class TestPersistentDirtyBitmap(iotests.QMPTestCase):
self.vm.shutdown()
+def test_reopen_rw(self):
+self.vm = self.mkVm()
+self.vm.launch()
+self.qmpAddBitmap()
+
+# Calculate hashes
+
+self.writeRegions(regions1)
+sha256_1 = self.getSha256()
+
+self.writeRegions(regions2)
+sha256_2 = self.getSha256()
+assert sha256_1 != sha256_2 # Otherwise, it's not very interesting.
+
+result = self.vm.qmp('block-dirty-bitmap-clear', node='drive0',
+ name='bitmap0')
+self.assert_qmp(result, 'return', {})
+
+# Start with regions1
+
+self.writeRegions(regions1)
+assert sha256_1 == self.getSha256()
+
+self.vm.shutdown()
+
+self.vm = self.mkVmRo()
+self.vm.launch()
+
+assert sha256_1 == self.getSha256()
+
+# Check that we are in RO mode and can't modify bitmap.
+self.writeRegions(regions2)
+assert sha256_1 == self.getSha256()
+
+# Reopen to RW
+result = self.vm.qmp('x-blockdev-reopen', **{
+'node-name': 'node0',
+'driver': iotests.imgfmt,
+'file': {
+'driver': 'file',
+'filename': disk
+},
+'read-only': False
+})
+self.assert_qmp(result, 'return', {})
+
+# Check that bitmap is reopened to RW and we can write to it.
+self.writeRegions(regions2)
+assert sha256_2 == self.getSha256()
+
+self.vm.shutdown()
+
+
if __name__ == '__main__':
iotests.main(supported_fmts=['qcow2'],
supported_protocols=['file'])
diff --git a/tests/qemu-iotests/165.out b/tests/qemu-iotests/165.out
index ae1213e6f86..fbc63e62f88 100644
--- a/tests/qemu-iotests/165.out
+++ b/tests/qemu-iotests/165.out
@@ -1,5 +1,5 @@
-.
+..
--
-Ran 1 tests
+Ran 2 tests
OK
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 08/19] block/dirty-bitmap: refactor bdrv_dirty_bitmap_next
From: Vladimir Sementsov-Ogievskiy
bdrv_dirty_bitmap_next is always used in same pattern. So, split it
into _next and _first, instead of combining two functions into one and
add FOR_EACH_DIRTY_BITMAP macro.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 9 +++--
block.c| 4 +---
block/dirty-bitmap.c | 11 +++
block/qcow2-bitmap.c | 8 ++--
migration/block-dirty-bitmap.c | 4 +---
5 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 2f9b088e11e..257f0f67046 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -96,8 +96,13 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap
*bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap);
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
+#define FOR_EACH_DIRTY_BITMAP(bs, bitmap) \
+for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
+ bitmap = bdrv_dirty_bitmap_next(bitmap))
+
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
uint64_t bytes);
diff --git a/block.c b/block.c
index bea03cfcc92..5b5b0337acc 100644
--- a/block.c
+++ b/block.c
@@ -5363,9 +5363,7 @@ static void coroutine_fn
bdrv_co_invalidate_cache(BlockDriverState *bs,
}
}
-for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm;
- bm = bdrv_dirty_bitmap_next(bs, bm))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bm) {
bdrv_dirty_bitmap_skip_store(bm, false);
}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 4e5c87a907f..6065db80949 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -851,11 +851,14 @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState
*bs)
return false;
}
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap)
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
-return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) :
-QLIST_NEXT(bitmap, list);
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap)
+{
+return QLIST_NEXT(bitmap, list);
}
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 687087d2bc2..99812b418b8 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1488,9 +1488,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
}
/* check constraints and names */
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
const char *name = bdrv_dirty_bitmap_name(bitmap);
uint32_t granularity = bdrv_dirty_bitmap_granularity(bitmap);
Qcow2Bitmap *bm;
@@ -1610,9 +1608,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
**errp)
return -EINVAL;
}
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (bdrv_dirty_bitmap_get_persistence(bitmap)) {
bdrv_dirty_bitmap_set_readonly(bitmap, true);
}
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 793f249aa5b..7eafface614 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -283,9 +283,7 @@ static int init_dirty_bitmap_migration(void)
for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
const char *name = bdrv_get_device_or_node_name(bs);
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (!bdrv_dirty_bitmap_name(bitmap)) {
continue;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 06/19] block/dirty-bitmap: add bs link
From: Vladimir Sementsov-Ogievskiy
Add bs field to BdrvDirtyBitmap structure. Drop BlockDriverState
parameter from bitmap APIs where possible.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-3-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 14 +-
block/backup.c | 14 ++
block/dirty-bitmap.c | 24
block/mirror.c | 4 ++--
block/qcow2-bitmap.c | 6 +++---
blockdev.c | 6 +++---
migration/block-dirty-bitmap.c | 7 +++
migration/block.c | 4 ++--
8 files changed, 36 insertions(+), 43 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 973056778aa..2f9b088e11e 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -18,21 +18,18 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
uint32_t granularity,
const char *name,
Error **errp);
-int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
- BdrvDirtyBitmap *bitmap,
+int bdrv_dirty_bitmap_create_successor(BdrvDirtyBitmap *bitmap,
Error **errp);
-BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap,
+BdrvDirtyBitmap *bdrv_dirty_bitmap_abdicate(BdrvDirtyBitmap *bitmap,
Error **errp);
-BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
- BdrvDirtyBitmap *bitmap,
+BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BdrvDirtyBitmap *bitmap,
Error **errp);
void bdrv_dirty_bitmap_enable_successor(BdrvDirtyBitmap *bitmap);
BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs,
const char *name);
int bdrv_dirty_bitmap_check(const BdrvDirtyBitmap *bitmap, uint32_t flags,
Error **errp);
-void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+void bdrv_release_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_release_named_dirty_bitmaps(BlockDriverState *bs);
int bdrv_remove_persistent_dirty_bitmap(BlockDriverState *bs, const char *name,
Error **errp);
@@ -106,8 +103,7 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap
*bitmap, uint64_t offset,
uint64_t bytes);
bool bdrv_dirty_bitmap_next_dirty_area(BdrvDirtyBitmap *bitmap,
uint64_t *offset, uint64_t *bytes);
-BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BlockDriverState *bs,
- BdrvDirtyBitmap *bitmap,
+BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap_locked(BdrvDirtyBitmap *bitmap,
Error **errp);
#endif
diff --git a/block/backup.c b/block/backup.c
index 763f0d7ff6d..acb67da3a7b 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -352,7 +352,6 @@ static int coroutine_fn backup_before_write_notify(
static void backup_cleanup_sync_bitmap(BackupBlockJob *job, int ret)
{
BdrvDirtyBitmap *bm;
-BlockDriverState *bs = blk_bs(job->common.blk);
bool sync = (((ret == 0) || (job->bitmap_mode == BITMAP_SYNC_MODE_ALWAYS))
\
&& (job->bitmap_mode != BITMAP_SYNC_MODE_NEVER));
@@ -361,13 +360,13 @@ static void backup_cleanup_sync_bitmap(BackupBlockJob
*job, int ret)
* We succeeded, or we always intended to sync the bitmap.
* Delete this bitmap and install the child.
*/
-bm = bdrv_dirty_bitmap_abdicate(bs, job->sync_bitmap, NULL);
+bm = bdrv_dirty_bitmap_abdicate(job->sync_bitmap, NULL);
} else {
/*
* We failed, or we never intended to sync the bitmap anyway.
* Merge the successor back into the parent, keeping all data.
*/
-bm = bdrv_reclaim_dirty_bitmap(bs, job->sync_bitmap, NULL);
+bm = bdrv_reclaim_dirty_bitmap(job->sync_bitmap, NULL);
}
assert(bm);
@@ -398,10 +397,9 @@ static void backup_abort(Job *job)
static void backup_clean(Job *job)
{
BackupBlockJob *s = container_of(job, BackupBlockJob, common.job);
-BlockDriverState *bs = blk_bs(s->common.blk);
if (s->copy_bitmap) {
-bdrv_release_dirty_bitmap(bs, s->copy_bitmap);
+bdrv_release_dirty_bitmap(s->copy_bitmap);
s->copy_bitmap = NULL;
}
@@ -679,7 +677,7 @@ BlockJob *backup_job_create(const char *job_id,
BlockDriverState *bs,
}
/* Create a new bitmap,
[libvirt] [PULL 09/19] block: switch reopen queue from QSIMPLEQ to QTAILQ
From: Vladimir Sementsov-Ogievskiy
We'll need reverse-foreach in the following commit, QTAILQ support it,
so move to QTAILQ.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Message-id: 20190927122355.7344-2-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/block.h | 2 +-
block.c | 24
2 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/include/block/block.h b/include/block/block.h
index 37c9de7446d..f5099435136 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -195,7 +195,7 @@ typedef struct HDGeometry {
#define BDRV_BLOCK_EOF 0x20
#define BDRV_BLOCK_RECURSE 0x40
-typedef QSIMPLEQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry)
BlockReopenQueue;
+typedef QTAILQ_HEAD(BlockReopenQueue, BlockReopenQueueEntry) BlockReopenQueue;
typedef struct BDRVReopenState {
BlockDriverState *bs;
diff --git a/block.c b/block.c
index 5b5b0337acc..aaf5d796284 100644
--- a/block.c
+++ b/block.c
@@ -1719,7 +1719,7 @@ typedef struct BlockReopenQueueEntry {
bool prepared;
bool perms_checked;
BDRVReopenState state;
- QSIMPLEQ_ENTRY(BlockReopenQueueEntry) entry;
+ QTAILQ_ENTRY(BlockReopenQueueEntry) entry;
} BlockReopenQueueEntry;
/*
@@ -1732,7 +1732,7 @@ static int bdrv_reopen_get_flags(BlockReopenQueue *q,
BlockDriverState *bs)
BlockReopenQueueEntry *entry;
if (q != NULL) {
-QSIMPLEQ_FOREACH(entry, q, entry) {
+QTAILQ_FOREACH(entry, q, entry) {
if (entry->state.bs == bs) {
return entry->state.flags;
}
@@ -3249,7 +3249,7 @@ static bool bdrv_recurse_has_child(BlockDriverState *bs,
* Adds a BlockDriverState to a simple queue for an atomic, transactional
* reopen of multiple devices.
*
- * bs_queue can either be an existing BlockReopenQueue that has had
QSIMPLE_INIT
+ * bs_queue can either be an existing BlockReopenQueue that has had QTAILQ_INIT
* already performed, or alternatively may be NULL a new BlockReopenQueue will
* be created and initialized. This newly created BlockReopenQueue should be
* passed back in for subsequent calls that are intended to be of the same
@@ -3290,7 +3290,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
if (bs_queue == NULL) {
bs_queue = g_new0(BlockReopenQueue, 1);
-QSIMPLEQ_INIT(bs_queue);
+QTAILQ_INIT(bs_queue);
}
if (!options) {
@@ -3298,7 +3298,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
}
/* Check if this BlockDriverState is already in the queue */
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
if (bs == bs_entry->state.bs) {
break;
}
@@ -3354,7 +3354,7 @@ static BlockReopenQueue
*bdrv_reopen_queue_child(BlockReopenQueue *bs_queue,
if (!bs_entry) {
bs_entry = g_new0(BlockReopenQueueEntry, 1);
-QSIMPLEQ_INSERT_TAIL(bs_queue, bs_entry, entry);
+QTAILQ_INSERT_TAIL(bs_queue, bs_entry, entry);
} else {
qobject_unref(bs_entry->state.options);
qobject_unref(bs_entry->state.explicit_options);
@@ -3455,7 +3455,7 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
assert(bs_queue != NULL);
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
assert(bs_entry->state.bs->quiesce_counter > 0);
if (bdrv_reopen_prepare(&bs_entry->state, bs_queue, errp)) {
goto cleanup;
@@ -3463,7 +3463,7 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
bs_entry->prepared = true;
}
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
BDRVReopenState *state = &bs_entry->state;
ret = bdrv_check_perm(state->bs, bs_queue, state->perm,
state->shared_perm, NULL, NULL, errp);
@@ -3489,13 +3489,13 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
/* If we reach this point, we have success and just need to apply the
* changes
*/
-QSIMPLEQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
bdrv_reopen_commit(&bs_entry->state);
}
ret = 0;
cleanup_perm:
-QSIMPLEQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
+QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
BDRVReopenState *state = &bs_entry->state;
if (!bs_entry->perms_checked) {
@@ -3512,7 +3512,7 @@ cleanup_perm:
}
}
cleanup:
-QSIMPLEQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
+QTAILQ_FOREACH_SAFE(bs_entry, bs_queue, entry, next) {
if (ret) {
if (bs_entry->prepared) {
[libvirt] [PULL 10/19] block: reverse order for reopen commits
From: Vladimir Sementsov-Ogievskiy
It's needed to fix reopening qcow2 with bitmaps to RW. Currently it
can't work, as qcow2 needs write access to file child, to mark bitmaps
in-image with IN_USE flag. But usually children goes after parents in
reopen queue and file child is still RO on qcow2 reopen commit. Reverse
reopen order to fix it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Acked-by: Max Reitz
Acked-by: John Snow
Message-id: 20190927122355.7344-3-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block.c | 12 +---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/block.c b/block.c
index aaf5d796284..c548885608d 100644
--- a/block.c
+++ b/block.c
@@ -3486,10 +3486,16 @@ int bdrv_reopen_multiple(BlockReopenQueue *bs_queue,
Error **errp)
bs_entry->perms_checked = true;
}
-/* If we reach this point, we have success and just need to apply the
- * changes
+/*
+ * If we reach this point, we have success and just need to apply the
+ * changes.
+ *
+ * Reverse order is used to comfort qcow2 driver: on commit it need to
write
+ * IN_USE flag to the image, to mark bitmaps in the image as invalid. But
+ * children are usually goes after parents in reopen-queue, so go from last
+ * to first element.
*/
-QTAILQ_FOREACH(bs_entry, bs_queue, entry) {
+QTAILQ_FOREACH_REVERSE(bs_entry, bs_queue, entry) {
bdrv_reopen_commit(&bs_entry->state);
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 12/19] block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps
From: Vladimir Sementsov-Ogievskiy
Firstly, no reason to optimize failure path. Then, function name is
ambiguous: it checks for readonly and similar things, but someone may
think that it will ignore normal bitmaps which was just unchanged, and
this is in bad relation with the fact that we should drop IN_USE flag
for unchanged bitmaps in the image.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 1 -
block/dirty-bitmap.c | 12
block/qcow2-bitmap.c | 23 +--
3 files changed, 13 insertions(+), 23 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 257f0f67046..958e7474fb5 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -95,7 +95,6 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 6065db80949..4bbb251b2c9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -839,18 +839,6 @@ bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap
*bitmap)
return bitmap->inconsistent;
}
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs)
-{
-BdrvDirtyBitmap *bm;
-QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
-if (bm->persistent && !bm->readonly && !bm->skip_store) {
-return true;
-}
-}
-
-return false;
-}
-
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
return QLIST_FIRST(&bs->dirty_bitmaps);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 99812b418b8..6dfc0835485 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1464,16 +1464,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
Qcow2Bitmap *bm;
QSIMPLEQ_HEAD(, Qcow2BitmapTable) drop_tables;
Qcow2BitmapTable *tb, *tb_next;
-
-if (!bdrv_has_changed_persistent_bitmaps(bs)) {
-/* nothing to do */
-return;
-}
-
-if (!can_write(bs)) {
-error_setg(errp, "No write access");
-return;
-}
+bool need_write = false;
QSIMPLEQ_INIT(&drop_tables);
@@ -1499,6 +1490,8 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
continue;
}
+need_write = true;
+
if (check_constraints_on_bitmap(bs, name, granularity, errp) < 0) {
error_prepend(errp, "Bitmap '%s' doesn't satisfy the constraints:
",
name);
@@ -1537,6 +1530,15 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bm->dirty_bitmap = bitmap;
}
+if (!need_write) {
+goto success;
+}
+
+if (!can_write(bs)) {
+error_setg(errp, "No write access");
+goto fail;
+}
+
/* allocate clusters and store bitmaps */
QSIMPLEQ_FOREACH(bm, bm_list, entry) {
if (bm->dirty_bitmap == NULL) {
@@ -1578,6 +1580,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bdrv_release_dirty_bitmap(bm->dirty_bitmap);
}
+success:
bitmap_list_free(bm_list);
return;
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 15/19] iotests: add test 260 to check bitmap life after snapshot + commit
From: Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-8-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
tests/qemu-iotests/260 | 89 ++
tests/qemu-iotests/260.out | 52 ++
tests/qemu-iotests/group | 1 +
3 files changed, 142 insertions(+)
create mode 100755 tests/qemu-iotests/260
create mode 100644 tests/qemu-iotests/260.out
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
new file mode 100755
index 000..4f6082c9d22
--- /dev/null
+++ b/tests/qemu-iotests/260
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Tests for temporary external snapshot when we have bitmaps.
+#
+# Copyright (c) 2019 Virtuozzo International GmbH. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import iotests
+from iotests import qemu_img_create, file_path, log, filter_qmp_event
+
+iotests.verify_image_format(supported_fmts=['qcow2'])
+
+base, top = file_path('base', 'top')
+size = 64 * 1024 * 3
+
+
+def print_bitmap(msg, vm):
+result = vm.qmp('query-block')['return'][0]
+if 'dirty-bitmaps' in result:
+bitmap = result['dirty-bitmaps'][0]
+log('{}: name={} dirty-clusters={}'.format(msg, bitmap['name'],
+bitmap['count'] // 64 // 1024))
+else:
+log(msg + ': not found')
+
+
+def test(persistent, restart):
+assert persistent or not restart
+log("\nTestcase {}persistent {} restart\n".format(
+'' if persistent else 'non-', 'with' if restart else 'without'))
+
+qemu_img_create('-f', iotests.imgfmt, base, str(size))
+
+vm = iotests.VM().add_drive(base)
+vm.launch()
+
+vm.qmp_log('block-dirty-bitmap-add', node='drive0', name='bitmap0',
+ persistent=persistent)
+vm.hmp_qemu_io('drive0', 'write 0 64K')
+print_bitmap('initial bitmap', vm)
+
+vm.qmp_log('blockdev-snapshot-sync', device='drive0', snapshot_file=top,
+ format=iotests.imgfmt, filters=[iotests.filter_qmp_testfiles])
+vm.hmp_qemu_io('drive0', 'write 64K 512')
+print_bitmap('check that no bitmaps are in snapshot', vm)
+
+if restart:
+log("... Restart ...")
+vm.shutdown()
+vm = iotests.VM().add_drive(top)
+vm.launch()
+
+vm.qmp_log('block-commit', device='drive0', top=top,
+ filters=[iotests.filter_qmp_testfiles])
+ev = vm.events_wait((('BLOCK_JOB_READY', None),
+ ('BLOCK_JOB_COMPLETED', None)))
+log(filter_qmp_event(ev))
+if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
+vm.shutdown()
+log(vm.get_log())
+exit()
+
+vm.qmp_log('block-job-complete', device='drive0')
+ev = vm.event_wait('BLOCK_JOB_COMPLETED')
+log(filter_qmp_event(ev))
+print_bitmap('check bitmap after commit', vm)
+
+vm.hmp_qemu_io('drive0', 'write 128K 64K')
+print_bitmap('check updated bitmap', vm)
+
+vm.shutdown()
+
+
+test(persistent=False, restart=False)
+test(persistent=True, restart=False)
+test(persistent=True, restart=True)
diff --git a/tests/qemu-iotests/260.out b/tests/qemu-iotests/260.out
new file mode 100644
index 000..2f0d98d0365
--- /dev/null
+++ b/tests/qemu-iotests/260.out
@@ -0,0 +1,52 @@
+
+Testcase non-persistent without restart
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"name": "bitmap0", "node":
"drive0", "persistent": false}}
+{"return": {}}
+initial bitmap: name=bitmap0 dirty-clusters=1
+{"execute": "blockdev-snapshot-sync", "arguments": {"device": "drive0",
"format": "qcow2", "snapshot-file": "TEST_DIR/PID-top"}}
+{"return": {}}
+check that no bitmaps are in snapshot: not found
+{"execute": "block-commit", "arguments&quo
[libvirt] [PULL 13/19] block/qcow2-bitmap: drop qcow2_reopen_bitmaps_rw_hint()
From: Vladimir Sementsov-Ogievskiy
The function is unused, drop it.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-6-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 2 --
block/qcow2-bitmap.c | 15 +--
2 files changed, 1 insertion(+), 16 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 113d99bf520..b3398a13c20 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -737,8 +737,6 @@ int qcow2_check_bitmaps_refcounts(BlockDriverState *bs,
BdrvCheckResult *res,
bool qcow2_load_dirty_bitmaps(BlockDriverState *bs, Error **errp);
Qcow2BitmapInfoList *qcow2_get_bitmap_info_list(BlockDriverState *bs,
Error **errp);
-int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
- Error **errp);
int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 6dfc0835485..ebc1afccd3d 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1102,8 +1102,7 @@ Qcow2BitmapInfoList
*qcow2_get_bitmap_info_list(BlockDriverState *bs,
return list;
}
-int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs, bool *header_updated,
- Error **errp)
+int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
{
BDRVQcow2State *s = bs->opaque;
Qcow2BitmapList *bm_list;
@@ -,10 +1110,6 @@ int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs,
bool *header_updated,
GSList *ro_dirty_bitmaps = NULL;
int ret = 0;
-if (header_updated != NULL) {
-*header_updated = false;
-}
-
if (s->nb_bitmaps == 0) {
/* No bitmaps - nothing to do */
return 0;
@@ -1156,9 +1151,6 @@ int qcow2_reopen_bitmaps_rw_hint(BlockDriverState *bs,
bool *header_updated,
error_setg_errno(errp, -ret, "Can't update bitmap directory");
goto out;
}
-if (header_updated != NULL) {
-*header_updated = true;
-}
g_slist_foreach(ro_dirty_bitmaps, set_readonly_helper, false);
}
@@ -1169,11 +1161,6 @@ out:
return ret;
}
-int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp)
-{
-return qcow2_reopen_bitmaps_rw_hint(bs, NULL, errp);
-}
-
/* Checks to see if it's safe to resize bitmaps */
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp)
{
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 18/19] MAINTAINERS: Add Vladimir as a reviewer for bitmaps
I already try to make sure all bitmaps patches have been reviewed by both Red Hat and Virtuozzo anyway, so this formalizes the arrangement. Fam meanwhile is no longer as active, so I am removing him as a co-maintainer simply to reflect the current practice. Signed-off-by: John Snow Reviewed-by: Vladimir Sementsov-Ogievskiy Message-id: 20191005194448.16629-2-js...@redhat.com --- MAINTAINERS | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/MAINTAINERS b/MAINTAINERS index 3ca814850e0..a08c92a4162 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1816,8 +1816,8 @@ F: qapi/transaction.json T: git https://repo.or.cz/qemu/armbru.git block-next Dirty Bitmaps -M: Fam Zheng M: John Snow +R: Vladimir Sementsov-Ogievskiy L: qemu-bl...@nongnu.org S: Supported F: util/hbitmap.c @@ -1826,7 +1826,6 @@ F: include/qemu/hbitmap.h F: include/block/dirty-bitmap.h F: tests/test-hbitmap.c F: docs/interop/bitmaps.rst -T: git https://github.com/famz/qemu.git bitmaps T: git https://github.com/jnsnow/qemu.git bitmaps Character device backends -- 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 16/19] block/qcow2-bitmap: fix and improve qcow2_reopen_bitmaps_rw
From: Vladimir Sementsov-Ogievskiy
- Correct check for write access to file child, and in correct place
(only if we want to write).
- Support reopen rw -> rw (which will be used in following commit),
for example, !bdrv_dirty_bitmap_readonly() is not a corruption if
bitmap is marked IN_USE in the image.
- Consider unexpected bitmap as a corruption and check other
combinations of in-image and in-RAM bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Message-id: 20190927122355.7344-9-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2-bitmap.c | 77 +---
1 file changed, 58 insertions(+), 19 deletions(-)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index f7dfb40256e..98294a76965 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1108,18 +1108,14 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp)
Qcow2BitmapList *bm_list;
Qcow2Bitmap *bm;
GSList *ro_dirty_bitmaps = NULL;
-int ret = 0;
+int ret = -EINVAL;
+bool need_header_update = false;
if (s->nb_bitmaps == 0) {
/* No bitmaps - nothing to do */
return 0;
}
-if (!can_write(bs)) {
-error_setg(errp, "Can't write to the image on reopening bitmaps rw");
-return -EINVAL;
-}
-
bm_list = bitmap_list_load(bs, s->bitmap_directory_offset,
s->bitmap_directory_size, errp);
if (bm_list == NULL) {
@@ -1128,32 +1124,75 @@ int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error
**errp)
QSIMPLEQ_FOREACH(bm, bm_list, entry) {
BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(bs, bm->name);
-if (bitmap == NULL) {
-continue;
-}
-if (!bdrv_dirty_bitmap_readonly(bitmap)) {
-error_setg(errp, "Bitmap %s was loaded prior to rw-reopen, but was
"
- "not marked as readonly. This is a bug, something went "
- "wrong. All of the bitmaps may be corrupted", bm->name);
-ret = -EINVAL;
+if (!bitmap) {
+error_setg(errp, "Unexpected bitmap '%s' in image '%s'",
+ bm->name, bs->filename);
goto out;
}
-bm->flags |= BME_FLAG_IN_USE;
-ro_dirty_bitmaps = g_slist_append(ro_dirty_bitmaps, bitmap);
+if (!(bm->flags & BME_FLAG_IN_USE)) {
+if (!bdrv_dirty_bitmap_readonly(bitmap)) {
+error_setg(errp, "Corruption: bitmap '%s' is not marked IN_USE
"
+ "in the image '%s' and not marked readonly in RAM",
+ bm->name, bs->filename);
+goto out;
+}
+if (bdrv_dirty_bitmap_inconsistent(bitmap)) {
+error_setg(errp, "Corruption: bitmap '%s' is inconsistent but "
+ "is not marked IN_USE in the image '%s'", bm->name,
+ bs->filename);
+goto out;
+}
+
+bm->flags |= BME_FLAG_IN_USE;
+need_header_update = true;
+} else {
+/*
+ * What if flags already has BME_FLAG_IN_USE ?
+ *
+ * 1. if we are reopening RW -> RW it's OK, of course.
+ * 2. if we are reopening RO -> RW:
+ * 2.1 if @bitmap is inconsistent, it's OK. It means that it was
+ * inconsistent (IN_USE) when we loaded it
+ * 2.2 if @bitmap is not inconsistent. This seems to be
impossible
+ * and implies third party interaction. Let's error-out for
+ * safety.
+ */
+if (bdrv_dirty_bitmap_readonly(bitmap) &&
+!bdrv_dirty_bitmap_inconsistent(bitmap))
+{
+error_setg(errp, "Corruption: bitmap '%s' is marked IN_USE "
+ "in the image '%s' but it is readonly and "
+ "consistent in RAM",
+ bm->name, bs->filename);
+goto out;
+}
+}
+
+if (bdrv_dirty_bitmap_readonly(bitmap)) {
+ro_dirty_bitmaps = g_slist_append(ro_dirty_bitmaps, bitmap);
+}
}
-if (ro_dirty_bitmaps != NULL) {
+if (need_header_update) {
+if (!can_write(bs->file->bs) || !(bs->file->perm & BLK_PERM_WRITE)) {
+error_setg(errp, "Failed to reopen bitmaps rw: no write access "
+ "the protocol file");
+goto out;
+}
+
/* in_use flags must be updated */
ret = upda
[libvirt] [PULL 17/19] qcow2-bitmap: move bitmap reopen-rw code to qcow2_reopen_commit
From: Vladimir Sementsov-Ogievskiy
The only reason I can imagine for this strange code at the very-end of
bdrv_reopen_commit is the fact that bs->read_only updated after
calling drv->bdrv_reopen_commit in bdrv_reopen_commit. And in the same
time, prior to previous commit, qcow2_reopen_bitmaps_rw did a wrong
check for being writable, when actually it only need writable file
child not self.
So, as it's fixed, let's move things to correct place.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Acked-by: Max Reitz
Message-id: 20190927122355.7344-10-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/block_int.h | 6 --
block.c | 19 ---
block/qcow2.c | 15 ++-
3 files changed, 14 insertions(+), 26 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 1e54486ad14..9ceca23ef75 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -546,12 +546,6 @@ struct BlockDriver {
uint64_t parent_perm, uint64_t parent_shared,
uint64_t *nperm, uint64_t *nshared);
-/**
- * Bitmaps should be marked as 'IN_USE' in the image on reopening image
- * as rw. This handler should realize it. It also should unset readonly
- * field of BlockDirtyBitmap's in case of success.
- */
-int (*bdrv_reopen_bitmaps_rw)(BlockDriverState *bs, Error **errp);
bool (*bdrv_co_can_store_new_dirty_bitmap)(BlockDriverState *bs,
const char *name,
uint32_t granularity,
diff --git a/block.c b/block.c
index c548885608d..ba09d97e0a2 100644
--- a/block.c
+++ b/block.c
@@ -3935,16 +3935,12 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state)
BlockDriver *drv;
BlockDriverState *bs;
BdrvChild *child;
-bool old_can_write, new_can_write;
assert(reopen_state != NULL);
bs = reopen_state->bs;
drv = bs->drv;
assert(drv != NULL);
-old_can_write =
-!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE);
-
/* If there are any driver level actions to take */
if (drv->bdrv_reopen_commit) {
drv->bdrv_reopen_commit(reopen_state);
@@ -3988,21 +3984,6 @@ void bdrv_reopen_commit(BDRVReopenState *reopen_state)
}
bdrv_refresh_limits(bs, NULL);
-
-new_can_write =
-!bdrv_is_read_only(bs) && !(bdrv_get_flags(bs) & BDRV_O_INACTIVE);
-if (!old_can_write && new_can_write && drv->bdrv_reopen_bitmaps_rw) {
-Error *local_err = NULL;
-if (drv->bdrv_reopen_bitmaps_rw(bs, &local_err) < 0) {
-/* This is not fatal, bitmaps just left read-only, so all following
- * writes will fail. User can remove read-only bitmaps to unblock
- * writes.
- */
-error_reportf_err(local_err,
- "%s: Failed to make dirty bitmaps writable: ",
- bdrv_get_node_name(bs));
-}
-}
}
/*
diff --git a/block/qcow2.c b/block/qcow2.c
index 7ab1aad9779..b652bf884e5 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1834,6 +1834,20 @@ fail:
static void qcow2_reopen_commit(BDRVReopenState *state)
{
qcow2_update_options_commit(state->bs, state->opaque);
+if (state->flags & BDRV_O_RDWR) {
+Error *local_err = NULL;
+
+if (qcow2_reopen_bitmaps_rw(state->bs, &local_err) < 0) {
+/*
+ * This is not fatal, bitmaps just left read-only, so all following
+ * writes will fail. User can remove read-only bitmaps to unblock
+ * writes or retry reopen.
+ */
+error_reportf_err(local_err,
+ "%s: Failed to make dirty bitmaps writable: ",
+ bdrv_get_node_name(state->bs));
+}
+}
g_free(state->opaque);
}
@@ -5262,7 +5276,6 @@ BlockDriver bdrv_qcow2 = {
.bdrv_detach_aio_context = qcow2_detach_aio_context,
.bdrv_attach_aio_context = qcow2_attach_aio_context,
-.bdrv_reopen_bitmaps_rw = qcow2_reopen_bitmaps_rw,
.bdrv_co_can_store_new_dirty_bitmap = qcow2_co_can_store_new_dirty_bitmap,
.bdrv_co_remove_persistent_dirty_bitmap =
qcow2_co_remove_persistent_dirty_bitmap,
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL 14/19] block/qcow2-bitmap: do not remove bitmaps on reopen-ro
From: Vladimir Sementsov-Ogievskiy
qcow2_reopen_bitmaps_ro wants to store bitmaps and then mark them all
readonly. But the latter don't work, as
qcow2_store_persistent_dirty_bitmaps removes bitmaps after storing.
It's OK for inactivation but bad idea for reopen-ro. And this leads to
the following bug:
Assume we have persistent bitmap 'bitmap0'.
Create external snapshot
bitmap0 is stored and therefore removed
Commit snapshot
now we have no bitmaps
Do some writes from guest (*)
they are not marked in bitmap
Shutdown
Start
bitmap0 is loaded as valid, but it is actually broken! It misses
writes (*)
Incremental backup
it will be inconsistent
So, let's stop removing bitmaps on reopen-ro. But don't rejoice:
reopening bitmaps to rw is broken too, so the whole scenario will not
work after this patch and we can't enable corresponding test cases in
260 iotests still. Reopening bitmaps rw will be fixed in the following
patches.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-7-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 3 ++-
block/qcow2-bitmap.c | 49 ++--
block/qcow2.c| 2 +-
3 files changed, 37 insertions(+), 17 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index b3398a13c20..8d293f2b64e 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -739,7 +739,8 @@ Qcow2BitmapInfoList
*qcow2_get_bitmap_info_list(BlockDriverState *bs,
Error **errp);
int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp);
int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
const char *name,
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index ebc1afccd3d..f7dfb40256e 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1440,7 +1440,32 @@ out:
return ret;
}
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
+/*
+ * qcow2_store_persistent_dirty_bitmaps
+ *
+ * Stores persistent BdrvDirtyBitmap objects.
+ *
+ * @release_stored: if true, release BdrvDirtyBitmap's after storing to the
+ * image. This is used in two cases, both via qcow2_inactivate:
+ * 1. bdrv_close: It's correct to remove bitmaps on close.
+ * 2. migration: If bitmaps are migrated through migration channel via
+ *'dirty-bitmaps' migration capability they are not handled by this code.
+ *Otherwise, it's OK to drop BdrvDirtyBitmap's and reload them on
+ *invalidation.
+ *
+ * Anyway, it's correct to remove BdrvDirtyBitmap's on inactivation, as
+ * inactivation means that we lose control on disk, and therefore on bitmaps,
+ * we should sync them and do not touch more.
+ *
+ * Contrariwise, we don't want to release any bitmaps on just reopen-to-ro,
+ * when we need to store them, as image is still under our control, and it's
+ * good to keep all the bitmaps in read-only mode. Moreover, keeping them
+ * read-only is correct because this is what would happen if we opened the node
+ * readonly to begin with, and whether we opened directly or reopened to that
+ * state shouldn't matter for the state we get afterward.
+ */
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp)
{
BdrvDirtyBitmap *bitmap;
BDRVQcow2State *s = bs->opaque;
@@ -1551,20 +1576,14 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
g_free(tb);
}
-QSIMPLEQ_FOREACH(bm, bm_list, entry) {
-/* For safety, we remove bitmap after storing.
- * We may be here in two cases:
- * 1. bdrv_close. It's ok to drop bitmap.
- * 2. inactivation. It means migration without 'dirty-bitmaps'
- *capability, so bitmaps are not marked with
- *BdrvDirtyBitmap.migration flags. It's not bad to drop them too,
- *and reload on invalidation.
- */
-if (bm->dirty_bitmap == NULL) {
-continue;
-}
+if (release_stored) {
+QSIMPLEQ_FOREACH(bm, bm_list, entry) {
+if (bm->dirty_bitmap == NULL) {
+continue;
+}
-bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+}
}
success:
@@ -1592,7 +1611,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
*
[libvirt] [PULL 19/19] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
Message-id: 20191002232411.29968-1-js...@redhat.com
---
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c4..7239e0959da 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 4.2.0)
+
+The "autoload" parameter has been ignored since 2.12.0. All bitmaps
+are automatically loaded from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e6edd641f18..e4975ece4ab 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1987,10 +1987,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -1999,7 +1995,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index 9b6eca66430..873aba3c27f 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PULL 01/19] util/hbitmap: strict hbitmap_reset
On 10/11/19 5:48 PM, Eric Blake wrote: > On 10/11/19 4:25 PM, John Snow wrote: >> From: Vladimir Sementsov-Ogievskiy >> >> hbitmap_reset has an unobvious property: it rounds requested region up. >> It may provoke bugs, like in recently fixed write-blocking mode of >> mirror: user calls reset on unaligned region, not keeping in mind that >> there are possible unrelated dirty bytes, covered by rounded-up region >> and information of this unrelated "dirtiness" will be lost. >> >> Make hbitmap_reset strict: assert that arguments are aligned, allowing >> only one exception when @start + @count == hb->orig_size. It's needed >> to comfort users of hbitmap_next_dirty_area, which cares about >> hb->orig_size. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> Reviewed-by: Max Reitz >> Message-Id: <20190806152611.280389-1-vsement...@virtuozzo.com> >> [Maintainer edit: Max's suggestions from on-list. --js] >> Signed-off-by: John Snow >> --- >> include/qemu/hbitmap.h | 5 + >> tests/test-hbitmap.c | 2 +- >> util/hbitmap.c | 4 >> 3 files changed, 10 insertions(+), 1 deletion(-) >> > >> +++ b/util/hbitmap.c >> @@ -476,6 +476,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, >> uint64_t count) >> /* Compute range in the last layer. */ >> uint64_t first; >> uint64_t last = start + count - 1; >> + uint64_t gran = 1ULL << hb->granularity; >> + >> + assert(!(start & (gran - 1))); >> + assert(!(count & (gran - 1)) || (start + count == hb->orig_size)); > > I know I'm replying a bit late (since this is now a pull request), but > would it be worth using the dedicated macro: > > assert(QEMU_IS_ALIGNED(start, gran)); > assert(QEMU_IS_ALIGNED(count, gran) || start + count == hb->orig_size); > > instead of open-coding it? (I would also drop the extra () around the > right half of ||). If we want it, that would now be a followup patch. > If the PR doesn't make it for some reason, I can amend a cleanup patch for the next PR. --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PULL 01/19] util/hbitmap: strict hbitmap_reset
On 10/11/19 7:18 PM, John Snow wrote: > > > On 10/11/19 5:48 PM, Eric Blake wrote: >> On 10/11/19 4:25 PM, John Snow wrote: >>> From: Vladimir Sementsov-Ogievskiy >>> >>> hbitmap_reset has an unobvious property: it rounds requested region up. >>> It may provoke bugs, like in recently fixed write-blocking mode of >>> mirror: user calls reset on unaligned region, not keeping in mind that >>> there are possible unrelated dirty bytes, covered by rounded-up region >>> and information of this unrelated "dirtiness" will be lost. >>> >>> Make hbitmap_reset strict: assert that arguments are aligned, allowing >>> only one exception when @start + @count == hb->orig_size. It's needed >>> to comfort users of hbitmap_next_dirty_area, which cares about >>> hb->orig_size. >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>> Reviewed-by: Max Reitz >>> Message-Id: <20190806152611.280389-1-vsement...@virtuozzo.com> >>> [Maintainer edit: Max's suggestions from on-list. --js] >>> Signed-off-by: John Snow >>> --- >>> include/qemu/hbitmap.h | 5 + >>> tests/test-hbitmap.c | 2 +- >>> util/hbitmap.c | 4 >>> 3 files changed, 10 insertions(+), 1 deletion(-) >>> >> >>> +++ b/util/hbitmap.c >>> @@ -476,6 +476,10 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, >>> uint64_t count) >>> /* Compute range in the last layer. */ >>> uint64_t first; >>> uint64_t last = start + count - 1; >>> + uint64_t gran = 1ULL << hb->granularity; >>> + >>> + assert(!(start & (gran - 1))); >>> + assert(!(count & (gran - 1)) || (start + count == hb->orig_size)); >> >> I know I'm replying a bit late (since this is now a pull request), but >> would it be worth using the dedicated macro: >> >> assert(QEMU_IS_ALIGNED(start, gran)); >> assert(QEMU_IS_ALIGNED(count, gran) || start + count == hb->orig_size); >> >> instead of open-coding it? (I would also drop the extra () around the >> right half of ||). If we want it, that would now be a followup patch. I've noticed that seasoned C programmers hate extra parentheses a lot. I've noticed that I cannot remember operator precedence enough to ever feel like this is actually an improvement. Something about a nice weighted tree of ((expr1) || (expr2)) feels soothing to my weary eyes. So, if it's not terribly important, I'd prefer to leave it as-is. (You may feel free to counter-educate me as desired.) >> > > If the PR doesn't make it for some reason, I can amend a cleanup patch > for the next PR. > by the way: GOOD NEWS! ... --js -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 12/19] block/qcow2-bitmap: get rid of bdrv_has_changed_persistent_bitmaps
From: Vladimir Sementsov-Ogievskiy
Firstly, no reason to optimize failure path. Then, function name is
ambiguous: it checks for readonly and similar things, but someone may
think that it will ignore normal bitmaps which was just unchanged, and
this is in bad relation with the fact that we should drop IN_USE flag
for unchanged bitmaps in the image.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 1 -
block/dirty-bitmap.c | 12
block/qcow2-bitmap.c | 23 +--
3 files changed, 13 insertions(+), 23 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 257f0f6704..958e7474fb 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -95,7 +95,6 @@ bool bdrv_has_readonly_bitmaps(BlockDriverState *bs);
bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 6065db8094..4bbb251b2c 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -839,18 +839,6 @@ bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap
*bitmap)
return bitmap->inconsistent;
}
-bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs)
-{
-BdrvDirtyBitmap *bm;
-QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
-if (bm->persistent && !bm->readonly && !bm->skip_store) {
-return true;
-}
-}
-
-return false;
-}
-
BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
return QLIST_FIRST(&bs->dirty_bitmaps);
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 99812b418b..6dfc083548 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1464,16 +1464,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
Qcow2Bitmap *bm;
QSIMPLEQ_HEAD(, Qcow2BitmapTable) drop_tables;
Qcow2BitmapTable *tb, *tb_next;
-
-if (!bdrv_has_changed_persistent_bitmaps(bs)) {
-/* nothing to do */
-return;
-}
-
-if (!can_write(bs)) {
-error_setg(errp, "No write access");
-return;
-}
+bool need_write = false;
QSIMPLEQ_INIT(&drop_tables);
@@ -1499,6 +1490,8 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
continue;
}
+need_write = true;
+
if (check_constraints_on_bitmap(bs, name, granularity, errp) < 0) {
error_prepend(errp, "Bitmap '%s' doesn't satisfy the constraints:
",
name);
@@ -1537,6 +1530,15 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bm->dirty_bitmap = bitmap;
}
+if (!need_write) {
+goto success;
+}
+
+if (!can_write(bs)) {
+error_setg(errp, "No write access");
+goto fail;
+}
+
/* allocate clusters and store bitmaps */
QSIMPLEQ_FOREACH(bm, bm_list, entry) {
if (bm->dirty_bitmap == NULL) {
@@ -1578,6 +1580,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
bdrv_release_dirty_bitmap(bm->dirty_bitmap);
}
+success:
bitmap_list_free(bm_list);
return;
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 19/19] dirty-bitmaps: remove deprecated autoload parameter
This parameter has been deprecated since 2.12.0 and is eligible for
removal. Remove this parameter as it is actually completely ignored;
let's not give false hope.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Vladimir Sementsov-Ogievskiy
Message-id: 20191002232411.29968-1-js...@redhat.com
---
qemu-deprecated.texi | 20 +++-
qapi/block-core.json | 6 +-
blockdev.c | 6 --
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/qemu-deprecated.texi b/qemu-deprecated.texi
index 01245e0b1c..7239e0959d 100644
--- a/qemu-deprecated.texi
+++ b/qemu-deprecated.texi
@@ -149,11 +149,6 @@ QEMU 4.1 has three options, please migrate to one of these
three:
@section QEMU Machine Protocol (QMP) commands
-@subsection block-dirty-bitmap-add "autoload" parameter (since 2.12.0)
-
-"autoload" parameter is now ignored. All bitmaps are automatically loaded
-from qcow2 images.
-
@subsection query-block result field dirty-bitmaps[i].status (since 4.0)
The ``status'' field of the ``BlockDirtyInfo'' structure, returned by
@@ -356,3 +351,18 @@ existing CPU models. Management software that needs
runnability
guarantees must resolve the CPU model aliases using te
``alias-of'' field returned by the ``query-cpu-definitions'' QMP
command.
+
+
+@node Recently removed features
+@appendix Recently removed features
+
+What follows is a record of recently removed, formerly deprecated
+features that serves as a record for users who have encountered
+trouble after a recent upgrade.
+
+@section QEMU Machine Protocol (QMP) commands
+
+@subsection block-dirty-bitmap-add "autoload" parameter (since 4.2.0)
+
+The "autoload" parameter has been ignored since 2.12.0. All bitmaps
+are automatically loaded from qcow2 images.
diff --git a/qapi/block-core.json b/qapi/block-core.json
index f66553aac7..b274aef713 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2052,10 +2052,6 @@
# Qcow2 disks support persistent bitmaps. Default is false for
# block-dirty-bitmap-add. (Since: 2.10)
#
-# @autoload: ignored and deprecated since 2.12.
-#Currently, all dirty tracking bitmaps are loaded from Qcow2 on
-#open.
-#
# @disabled: the bitmap is created in the disabled state, which means that
#it will not track drive changes. The bitmap may be enabled with
#block-dirty-bitmap-enable. Default is false. (Since: 4.0)
@@ -2064,7 +2060,7 @@
##
{ 'struct': 'BlockDirtyBitmapAdd',
'data': { 'node': 'str', 'name': 'str', '*granularity': 'uint32',
-'*persistent': 'bool', '*autoload': 'bool', '*disabled': 'bool' } }
+'*persistent': 'bool', '*disabled': 'bool' } }
##
# @BlockDirtyBitmapMergeSource:
diff --git a/blockdev.c b/blockdev.c
index d77e809623..03c7cd7651 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1966,7 +1966,6 @@ static void block_dirty_bitmap_add_prepare(BlkActionState
*common,
qmp_block_dirty_bitmap_add(action->node, action->name,
action->has_granularity, action->granularity,
action->has_persistent, action->persistent,
- action->has_autoload, action->autoload,
action->has_disabled, action->disabled,
&local_err);
@@ -2858,7 +2857,6 @@ out:
void qmp_block_dirty_bitmap_add(const char *node, const char *name,
bool has_granularity, uint32_t granularity,
bool has_persistent, bool persistent,
-bool has_autoload, bool autoload,
bool has_disabled, bool disabled,
Error **errp)
{
@@ -2890,10 +2888,6 @@ void qmp_block_dirty_bitmap_add(const char *node, const
char *name,
persistent = false;
}
-if (has_autoload) {
-warn_report("Autoload option is deprecated and its value is ignored");
-}
-
if (!has_disabled) {
disabled = false;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 08/19] block/dirty-bitmap: refactor bdrv_dirty_bitmap_next
From: Vladimir Sementsov-Ogievskiy
bdrv_dirty_bitmap_next is always used in same pattern. So, split it
into _next and _first, instead of combining two functions into one and
add FOR_EACH_DIRTY_BITMAP macro.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190916141911.5255-5-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
include/block/dirty-bitmap.h | 9 +++--
block.c| 4 +---
block/dirty-bitmap.c | 11 +++
block/qcow2-bitmap.c | 8 ++--
migration/block-dirty-bitmap.c | 4 +---
5 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 2f9b088e11..257f0f6704 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -96,8 +96,13 @@ bool bdrv_dirty_bitmap_get_autoload(const BdrvDirtyBitmap
*bitmap);
bool bdrv_dirty_bitmap_get_persistence(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_inconsistent(const BdrvDirtyBitmap *bitmap);
bool bdrv_has_changed_persistent_bitmaps(BlockDriverState *bs);
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap);
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs);
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap);
+#define FOR_EACH_DIRTY_BITMAP(bs, bitmap) \
+for (bitmap = bdrv_dirty_bitmap_first(bs); bitmap; \
+ bitmap = bdrv_dirty_bitmap_next(bitmap))
+
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t offset,
uint64_t bytes);
diff --git a/block.c b/block.c
index d19a4781a3..5721441697 100644
--- a/block.c
+++ b/block.c
@@ -5390,9 +5390,7 @@ static void coroutine_fn
bdrv_co_invalidate_cache(BlockDriverState *bs,
}
}
-for (bm = bdrv_dirty_bitmap_next(bs, NULL); bm;
- bm = bdrv_dirty_bitmap_next(bs, bm))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bm) {
bdrv_dirty_bitmap_skip_store(bm, false);
}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 4e5c87a907..6065db8094 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -851,11 +851,14 @@ bool bdrv_has_changed_persistent_bitmaps(BlockDriverState
*bs)
return false;
}
-BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
-BdrvDirtyBitmap *bitmap)
+BdrvDirtyBitmap *bdrv_dirty_bitmap_first(BlockDriverState *bs)
{
-return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) :
-QLIST_NEXT(bitmap, list);
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BdrvDirtyBitmap *bitmap)
+{
+return QLIST_NEXT(bitmap, list);
}
char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index 687087d2bc..99812b418b 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1488,9 +1488,7 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
}
/* check constraints and names */
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
const char *name = bdrv_dirty_bitmap_name(bitmap);
uint32_t granularity = bdrv_dirty_bitmap_granularity(bitmap);
Qcow2Bitmap *bm;
@@ -1610,9 +1608,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
**errp)
return -EINVAL;
}
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap != NULL;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (bdrv_dirty_bitmap_get_persistence(bitmap)) {
bdrv_dirty_bitmap_set_readonly(bitmap, true);
}
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 793f249aa5..7eafface61 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -283,9 +283,7 @@ static int init_dirty_bitmap_migration(void)
for (bs = bdrv_next_all_states(NULL); bs; bs = bdrv_next_all_states(bs)) {
const char *name = bdrv_get_device_or_node_name(bs);
-for (bitmap = bdrv_dirty_bitmap_next(bs, NULL); bitmap;
- bitmap = bdrv_dirty_bitmap_next(bs, bitmap))
-{
+FOR_EACH_DIRTY_BITMAP(bs, bitmap) {
if (!bdrv_dirty_bitmap_name(bitmap)) {
continue;
}
--
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
[libvirt] [PULL v2 14/19] block/qcow2-bitmap: do not remove bitmaps on reopen-ro
From: Vladimir Sementsov-Ogievskiy
qcow2_reopen_bitmaps_ro wants to store bitmaps and then mark them all
readonly. But the latter don't work, as
qcow2_store_persistent_dirty_bitmaps removes bitmaps after storing.
It's OK for inactivation but bad idea for reopen-ro. And this leads to
the following bug:
Assume we have persistent bitmap 'bitmap0'.
Create external snapshot
bitmap0 is stored and therefore removed
Commit snapshot
now we have no bitmaps
Do some writes from guest (*)
they are not marked in bitmap
Shutdown
Start
bitmap0 is loaded as valid, but it is actually broken! It misses
writes (*)
Incremental backup
it will be inconsistent
So, let's stop removing bitmaps on reopen-ro. But don't rejoice:
reopening bitmaps to rw is broken too, so the whole scenario will not
work after this patch and we can't enable corresponding test cases in
260 iotests still. Reopening bitmaps rw will be fixed in the following
patches.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Message-id: 20190927122355.7344-7-vsement...@virtuozzo.com
Signed-off-by: John Snow
---
block/qcow2.h| 3 ++-
block/qcow2-bitmap.c | 49 ++--
block/qcow2.c| 2 +-
3 files changed, 37 insertions(+), 17 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index 23a9898a54..5cccd87162 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -742,7 +742,8 @@ Qcow2BitmapInfoList
*qcow2_get_bitmap_info_list(BlockDriverState *bs,
Error **errp);
int qcow2_reopen_bitmaps_rw(BlockDriverState *bs, Error **errp);
int qcow2_truncate_bitmaps_check(BlockDriverState *bs, Error **errp);
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp);
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp);
int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error **errp);
bool qcow2_co_can_store_new_dirty_bitmap(BlockDriverState *bs,
const char *name,
diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c
index ebc1afccd3..f7dfb40256 100644
--- a/block/qcow2-bitmap.c
+++ b/block/qcow2-bitmap.c
@@ -1440,7 +1440,32 @@ out:
return ret;
}
-void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
+/*
+ * qcow2_store_persistent_dirty_bitmaps
+ *
+ * Stores persistent BdrvDirtyBitmap objects.
+ *
+ * @release_stored: if true, release BdrvDirtyBitmap's after storing to the
+ * image. This is used in two cases, both via qcow2_inactivate:
+ * 1. bdrv_close: It's correct to remove bitmaps on close.
+ * 2. migration: If bitmaps are migrated through migration channel via
+ *'dirty-bitmaps' migration capability they are not handled by this code.
+ *Otherwise, it's OK to drop BdrvDirtyBitmap's and reload them on
+ *invalidation.
+ *
+ * Anyway, it's correct to remove BdrvDirtyBitmap's on inactivation, as
+ * inactivation means that we lose control on disk, and therefore on bitmaps,
+ * we should sync them and do not touch more.
+ *
+ * Contrariwise, we don't want to release any bitmaps on just reopen-to-ro,
+ * when we need to store them, as image is still under our control, and it's
+ * good to keep all the bitmaps in read-only mode. Moreover, keeping them
+ * read-only is correct because this is what would happen if we opened the node
+ * readonly to begin with, and whether we opened directly or reopened to that
+ * state shouldn't matter for the state we get afterward.
+ */
+void qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs,
+ bool release_stored, Error **errp)
{
BdrvDirtyBitmap *bitmap;
BDRVQcow2State *s = bs->opaque;
@@ -1551,20 +1576,14 @@ void
qcow2_store_persistent_dirty_bitmaps(BlockDriverState *bs, Error **errp)
g_free(tb);
}
-QSIMPLEQ_FOREACH(bm, bm_list, entry) {
-/* For safety, we remove bitmap after storing.
- * We may be here in two cases:
- * 1. bdrv_close. It's ok to drop bitmap.
- * 2. inactivation. It means migration without 'dirty-bitmaps'
- *capability, so bitmaps are not marked with
- *BdrvDirtyBitmap.migration flags. It's not bad to drop them too,
- *and reload on invalidation.
- */
-if (bm->dirty_bitmap == NULL) {
-continue;
-}
+if (release_stored) {
+QSIMPLEQ_FOREACH(bm, bm_list, entry) {
+if (bm->dirty_bitmap == NULL) {
+continue;
+}
-bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+bdrv_release_dirty_bitmap(bm->dirty_bitmap);
+}
}
success:
@@ -1592,7 +1611,7 @@ int qcow2_reopen_bitmaps_ro(BlockDriverState *bs, Error
*
