Re: [Qemu-devel] [PATCH RFC v2 4/8] block: add dirty-dirty bitmaps
On 11.02.2015 00:30, John Snow wrote:
I had hoped it wouldn't come to this :)
On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote:
A dirty-dirty bitmap is a dirty bitmap for BdrvDirtyBitmap. It tracks
set/unset changes of BdrvDirtyBitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 44
include/block/block.h | 5 +
2 files changed, 49 insertions(+)
diff --git a/block.c b/block.c
index 9860fc1..8ab724b 100644
--- a/block.c
+++ b/block.c
@@ -53,6 +53,7 @@
struct BdrvDirtyBitmap {
HBitmap *bitmap;
+HBitmap *dirty_dirty_bitmap;
Just opinions: Maybe we can call it the "meta_bitmap" to help keep the
name less confusing, and accompany it with a good structure comment
here to explain what the heck it's for.
If you feel that's a good idea; s/dirty_dirty_/meta_/g below.
Regardless, let's make sure this patch adds documentation for it.
No objections. If everyone is happy with meta_bitmaps - I'll use this
name. Dirty-dirty bitmaps are just more fanny, I think ;)
BdrvDirtyBitmap *originator;
int64_t size;
int64_t granularity;
@@ -5373,6 +5374,30 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
return originator;
}
+HBitmap *bdrv_create_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity)
+{
+uint64_t sector_granularity;
+
+assert((granularity & (granularity - 1)) == 0);
+
+granularity *= 8 * bitmap->granularity;
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+
+bitmap->dirty_dirty_bitmap =
+hbitmap_alloc(bitmap->size, ffsll(sector_granularity) - 1);
+
+return bitmap->dirty_dirty_bitmap;
+}
+
+void bdrv_release_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_free(bitmap->dirty_dirty_bitmap);
+bitmap->dirty_dirty_bitmap = NULL;
+}
+}
void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
@@ -5447,6 +5472,9 @@ void bdrv_release_dirty_bitmap(BlockDriverState
*bs, BdrvDirtyBitmap *bitmap)
if (bm == bitmap) {
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_free(bitmap->dirty_dirty_bitmap);
+}
g_free(bitmap->name);
g_free(bitmap);
return;
@@ -5534,6 +5562,10 @@ void bdrv_set_dirty_bitmap(BlockDriverState
*bs, BdrvDirtyBitmap *bitmap,
{
if (bitmap->enabled) {
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector,
nr_sectors);
+}
}
}
@@ -5541,6 +5573,9 @@ void bdrv_reset_dirty_bitmap(BlockDriverState
*bs, BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors)
{
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector,
nr_sectors);
+}
}
/**
@@ -5550,6 +5585,9 @@ void bdrv_reset_dirty_bitmap(BlockDriverState
*bs, BdrvDirtyBitmap *bitmap,
void bdrv_clear_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap)
{
hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, 0, bitmap->size);
+}
}
const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
@@ -5597,6 +5635,9 @@ static void bdrv_set_dirty(BlockDriverState
*bs, int64_t cur_sector,
continue;
}
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector,
nr_sectors);
+}
}
}
@@ -5606,6 +5647,9 @@ static void bdrv_reset_dirty(BlockDriverState
*bs, int64_t cur_sector,
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector,
nr_sectors);
+}
}
}
diff --git a/include/block/block.h b/include/block/block.h
index 0890cd2..648b0a9 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,6 +4,7 @@
#include "block/aio.h"
#include "qemu-common.h"
#include "qemu/option.h"
+#include "qemu/hbitmap.h"
#include "block/coroutine.h"
#include "block/accounting.h"
#include "qapi/qmp/qobject.h"
@@ -473,6 +474,10 @@ void
bdrv_dirt
Re: [Qemu-devel] [PATCH RFC v2 5/8] block: add bdrv_dirty_bitmap_enabled()
On 11.02.2015 00:30, John Snow wrote:
On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote:
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 5 +
include/block/block.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/block.c b/block.c
index 8ab724b..15fc621 100644
--- a/block.c
+++ b/block.c
@@ -5551,6 +5551,11 @@ uint64_t
bdrv_dirty_bitmap_granularity(BlockDriverState *bs,
return bitmap->granularity;
}
+bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
+{
+return bitmap->enabled;
+}
+
void bdrv_dirty_iter_init(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
{
diff --git a/include/block/block.h b/include/block/block.h
index 648b0a9..7b49d98 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -449,6 +449,7 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs);
uint64_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
uint64_t bdrv_dirty_bitmap_granularity(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap);
+bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap);
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
int64_t sector);
void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap,
int64_t cur_sector, int nr_sectors);
I wrote something similar in my incremental backup series. I'm
submitting a new incremental backup series (v12!) soon which might
have /slightly/ different semantics for enabled/disabled bitmaps.
I don't think you'll need this patch, but in case you do need it:
Reviewed-by: John Snow
Ok, I don't need it after v12.
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
On 11.02.2015 00:33, John Snow wrote:
Peter Maydell: What's the right way to license a file as copied from a
previous version? See below, please;
Max, Markus: ctrl+f "bdrv_get_device_name" and let me know what you
think, if you would.
Juan, Amit, David: Copying migration maintainers.
On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote:
Live migration of dirty bitmaps. Only named dirty bitmaps are migrated.
If destination qemu is already containing a dirty bitmap with the same
name as a migrated bitmap, then their granularities should be the same,
otherwise the error will be generated. If destination qemu doesn't
contain such bitmap it will be created.
format:
1 byte: flags
[ 1 byte: node name size ] \ flags & DEVICE_NAME
[ n bytes: node name ] /
[ 1 byte: bitmap name size ] \
[ n bytes: bitmap name ] | flags & BITMAP_NAME
[ [ be64: granularity] ] flags & GRANULARITY
[ 1 byte: bitmap enabled bit ] flags & ENABLED
[ be64: start sector ] \ flags & (NORMAL_CHUNK | ZERO_CHUNK)
[ be32: number of sectors ] /
[ be64: buffer size ] \ flags & NORMAL_CHUNK
[ n bytes: buffer ] /
The last chunk should contain flags & EOS. The chunk may skip device
and/or bitmap names, assuming them to be the same with the previous
chunk. GRANULARITY is sent with the first chunk for the bitmap. ENABLED
bit is sent in the end of "complete" stage of migration. So when
destination gets ENABLED flag it should deserialize_finish the bitmap
and set its enabled bit to corresponding value.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/block.h | 1 +
migration/Makefile.objs | 2 +-
migration/dirty-bitmap.c | 606
++
vl.c | 1 +
4 files changed, 609 insertions(+), 1 deletion(-)
create mode 100644 migration/dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
OK.
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..9adfda9 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o dirty-bitmap.o
OK.
diff --git a/migration/dirty-bitmap.c b/migration/dirty-bitmap.c
new file mode 100644
index 000..8621218
--- /dev/null
+++ b/migration/dirty-bitmap.c
@@ -0,0 +1,606 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * derived from migration/block.c
+ *
+ * Author:
+ * Sementsov-Ogievskiy Vladimir
+ *
+ * original copyright message:
+ *
=
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ *
=
+ */
+
Not super familiar with the right way to do licensing here; it's
possible you may not need to copy the original here, but I'm not sure.
You will want to make it clear what license applies to /your/ work, I
think. Maybe Peter Maydell can clue us in.
+#include "block/block.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include
+
+#define CHUNK_SIZE (1 << 20)
+
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_NORMAL_CHUNK 0x02
+#define DIRTY_BITMAP_MIG_FLAG_ZERO_CHUNK0x04
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x10
+#define DIRTY_BITMAP_MIG_FLAG_GRANULARITY 0x20
+#define DIRTY_BITMAP_MIG_FLAG_ENABLED 0x40
+/* flags should be <= 0xff */
+
We should give ourselves a little breathing room with the flags, since
we've only got room for one more.
Ok. Will one more byte be enough?
+/* #define DEBUG_DIRTY_BITMAP_MIGRATION */
+
+#ifdef DEBUG_DIRTY_BITMAP_MIGRATION
+#define DPRINTF(fmt, ...) \
+do { printf("dirty_migration: " fmt, ## __VA_ARGS__); } while (0)
+#else
+#define DPRINTF(fmt, ...) \
+do { } while (0)
+#endif
+
+typedef struct DirtyBitmapMigBitmapState {
+/* Written during setup phase. */
+BlockDriverState *bs;
+BdrvDirtyBitmap
Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
+
+blk_mig_reset_dirty_cursor();
+dirty_phase(f, false);
+
+QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list, entry) {
+uint8_t flags = DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME |
+DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME |
+DIRTY_BITMAP_MIG_FLAG_ENABLED;
+
+qemu_put_byte(f, flags);
+qemu_put_name(f, bdrv_get_device_name(dbms->bs));
+qemu_put_name(f, bdrv_dirty_bitmap_name(dbms->bitmap));
+qemu_put_byte(f, bdrv_dirty_bitmap_enabled(dbms->bitmap));
+}
+
+qemu_put_byte(f, DIRTY_BITMAP_MIG_FLAG_EOS);
+
+DPRINTF("Dirty bitmaps migration completed\n");
+
+dirty_bitmap_mig_cleanup();
+return 0;
+}
+
I suppose we don't need a flag that distinctly SAYS this is the end
section, since we can tell by omission of
DIRTY_BITMAP_MIG_FLAG_NORMAL_CHUNK or ZERO_CHUNK.
Hmm. I think it simplifies the logic (to use EOS after each section).
And the same approach is in migration/block.c.. It's a question about
which format is better: "Each section for dirty_bitmap_load ends with
EOS" or "Each section for dirty_bitmap_load ends with EOS except the
last one. The last one may be recognized by absent NORMAL_CHUNK and
ZERO_CHUNK"
Oh, sorry, no, it's important EOS. There are several blocks with no
*_CHUNK! Several bitmaps. And loop in dirty_bitmap_load will read them
iteratively, and it will finish when find EOS.
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH v12 07/17] qmp: Add support of "dirty-bitmap" sync mode for drive-backup
On 10.02.2015 04:35, John Snow wrote:
..
@@ -278,28 +305,61 @@ static void coroutine_fn backup_run(void *opaque)
qemu_coroutine_yield();
job->common.busy = true;
}
+} else if (job->sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) {
+/* Dirty Bitmap sync has a slightly different iteration method */
+HBitmapIter hbi;
+int64_t sector;
+int64_t cluster;
+int64_t last_cluster = -1;
+bool polyrhythmic;
+
+bdrv_dirty_iter_init(bs, job->sync_bitmap, &hbi);
+/* Does the granularity happen to match our backup cluster size? */
+polyrhythmic = (bdrv_dirty_bitmap_granularity(job->sync_bitmap) !=
+BACKUP_CLUSTER_SIZE);
let it be false, i.e. granularity == cluster
+
+/* Find the next dirty /sector/ and copy that /cluster/ */
+while ((sector = hbitmap_iter_next(&hbi)) != -1) {
then, don't we skip here the very first cluster, if it is dirty?
+if (yield_and_check(job)) {
+goto leave;
+}
+cluster = sector / BACKUP_SECTORS_PER_CLUSTER;
+
+/* Play some catchup with the progress meter */
+if (cluster != last_cluster + 1) {
+job->common.offset += ((cluster - last_cluster - 1) *
+ BACKUP_CLUSTER_SIZE);
+}
+
+do {
+ret = backup_do_cow(bs, cluster * BACKUP_SECTORS_PER_CLUSTER,
+BACKUP_SECTORS_PER_CLUSTER,
&error_is_read);
+if ((ret < 0) &&
+backup_error_action(job, error_is_read, -ret) ==
+BLOCK_ERROR_ACTION_REPORT) {
+goto leave;
+}
+} while (ret < 0);
+
+/* Advance (or rewind) our iterator if we need to. */
+if (polyrhythmic) {
+bdrv_set_dirty_iter(&hbi,
+(cluster + 1) *
BACKUP_SECTORS_PER_CLUSTER);
+}
+
+last_cluster = cluster;
+}
+
+/* Play some final catchup with the progress meter */
+if (last_cluster + 1 < end) {
+job->common.offset += ((end - last_cluster - 1) *
+ BACKUP_CLUSTER_SIZE);
+}
+
} else {
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
On 13.02.2015 20:32, John Snow wrote:
On 02/13/2015 04:06 AM, Vladimir Sementsov-Ogievskiy wrote:
+
+blk_mig_reset_dirty_cursor();
+dirty_phase(f, false);
+
+QSIMPLEQ_FOREACH(dbms, &dirty_bitmap_mig_state.dbms_list,
entry) {
+uint8_t flags = DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME |
+DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME |
+DIRTY_BITMAP_MIG_FLAG_ENABLED;
+
+qemu_put_byte(f, flags);
+qemu_put_name(f, bdrv_get_device_name(dbms->bs));
+qemu_put_name(f, bdrv_dirty_bitmap_name(dbms->bitmap));
+qemu_put_byte(f, bdrv_dirty_bitmap_enabled(dbms->bitmap));
+}
+
+qemu_put_byte(f, DIRTY_BITMAP_MIG_FLAG_EOS);
+
+DPRINTF("Dirty bitmaps migration completed\n");
+
+dirty_bitmap_mig_cleanup();
+return 0;
+}
+
I suppose we don't need a flag that distinctly SAYS this is the end
section, since we can tell by omission of
DIRTY_BITMAP_MIG_FLAG_NORMAL_CHUNK or ZERO_CHUNK.
Hmm. I think it simplifies the logic (to use EOS after each section).
And the same approach is in migration/block.c.. It's a question about
which format is better: "Each section for dirty_bitmap_load ends with
EOS" or "Each section for dirty_bitmap_load ends with EOS except the
last one. The last one may be recognized by absent NORMAL_CHUNK and
ZERO_CHUNK"
Oh, sorry, no, it's important EOS. There are several blocks with no
*_CHUNK! Several bitmaps. And loop in dirty_bitmap_load will read them
iteratively, and it will finish when find EOS.
Sorry, I worded that poorly. I was wondering why you didn't have an
explicit "end of bitmap" flag, and I realized that you are doing this
check essentially by the absence of the NORMAL_CHUNK/ZERO_CHUNK flags.
This is really just a comment on my part; I was expecting a more
distinct "It is now safe to rebuild the bitmap" flag and was just
commenting on why we didn't necessarily need one.
I think in another comment I point out that an "end of bitmap" flag
might make the loading function simpler, and I still think that might
be nice.
Ok. Today I've refactored these things, there would be separate start
and complete frames of bitmap, the first with additional granularity
field (without any dirty bitmap data) and the second with additional
enabled field (also without data).
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
On 13.02.2015 23:22, John Snow wrote: On 02/13/2015 03:19 AM, Vladimir Sementsov-Ogievskiy wrote: On 11.02.2015 00:33, John Snow wrote: Peter Maydell: What's the right way to license a file as copied from a previous version? See below, please; Max, Markus: ctrl+f "bdrv_get_device_name" and let me know what you think, if you would. Juan, Amit, David: Copying migration maintainers. On 01/27/2015 05:56 AM, Vladimir Sementsov-Ogievskiy wrote: Live migration of dirty bitmaps. Only named dirty bitmaps are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap, then their granularities should be the same, otherwise the error will be generated. If destination qemu doesn't contain such bitmap it will be created. format: 1 byte: flags [ 1 byte: node name size ] \ flags & DEVICE_NAME [ n bytes: node name ] / [ 1 byte: bitmap name size ] \ [ n bytes: bitmap name ] | flags & BITMAP_NAME [ [ be64: granularity] ] flags & GRANULARITY [ 1 byte: bitmap enabled bit ] flags & ENABLED [ be64: start sector ] \ flags & (NORMAL_CHUNK | ZERO_CHUNK) [ be32: number of sectors ] / [ be64: buffer size ] \ flags & NORMAL_CHUNK [ n bytes: buffer ] / The last chunk should contain flags & EOS. The chunk may skip device and/or bitmap names, assuming them to be the same with the previous chunk. GRANULARITY is sent with the first chunk for the bitmap. ENABLED bit is sent in the end of "complete" stage of migration. So when destination gets ENABLED flag it should deserialize_finish the bitmap and set its enabled bit to corresponding value. Signed-off-by: Vladimir Sementsov-Ogievskiy --- include/migration/block.h | 1 + migration/Makefile.objs | 2 +- migration/dirty-bitmap.c | 606 ++ vl.c | 1 + 4 files changed, 609 insertions(+), 1 deletion(-) create mode 100644 migration/dirty-bitmap.c diff --git a/include/migration/block.h b/include/migration/block.h index ffa8ac0..566bb9f 100644 --- a/include/migration/block.h +++ b/include/migration/block.h @@ -14,6 +14,7 @@ #ifndef BLOCK_MIGRATION_H #define BLOCK_MIGRATION_H +void dirty_bitmap_mig_init(void); void blk_mig_init(void); int blk_mig_active(void); uint64_t blk_mig_bytes_transferred(void); OK. diff --git a/migration/Makefile.objs b/migration/Makefile.objs index d929e96..9adfda9 100644 --- a/migration/Makefile.objs +++ b/migration/Makefile.objs @@ -6,5 +6,5 @@ common-obj-y += xbzrle.o common-obj-$(CONFIG_RDMA) += rdma.o common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o -common-obj-y += block.o +common-obj-y += block.o dirty-bitmap.o OK. diff --git a/migration/dirty-bitmap.c b/migration/dirty-bitmap.c new file mode 100644 index 000..8621218 --- /dev/null +++ b/migration/dirty-bitmap.c @@ -0,0 +1,606 @@ +/* + * QEMU dirty bitmap migration + * + * derived from migration/block.c + * + * Author: + * Sementsov-Ogievskiy Vladimir + * + * original copyright message: + * = + * Copyright IBM, Corp. 2009 + * + * Authors: + * Liran Schour + * + * This work is licensed under the terms of the GNU GPL, version 2. See + * the COPYING file in the top-level directory. + * + * Contributions after 2012-01-13 are licensed under the terms of the + * GNU GPL, version 2 or (at your option) any later version. + * = + */ + Not super familiar with the right way to do licensing here; it's possible you may not need to copy the original here, but I'm not sure. You will want to make it clear what license applies to /your/ work, I think. Maybe Peter Maydell can clue us in. +#include "block/block.h" +#include "qemu/main-loop.h" +#include "qemu/error-report.h" +#include "migration/block.h" +#include "migration/migration.h" +#include "qemu/hbitmap.h" +#include + +#define CHUNK_SIZE (1 << 20) + +#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01 +#define DIRTY_BITMAP_MIG_FLAG_NORMAL_CHUNK 0x02 +#define DIRTY_BITMAP_MIG_FLAG_ZERO_CHUNK0x04 +#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x08 +#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x10 +#define DIRTY_BITMAP_MIG_FLAG_GRANULARITY 0x20 +#define DIRTY_BITMAP_MIG_FLAG_ENABLED 0x40 +/* flags should be <= 0xff */ + We should give ourselves a little breathing room with the flags, since we've only got room for one more. Ok. Will one more byte be enough? I should hope so. If you do add a completion chunk and flag, that fills up the first byte completely, so having a second byte is a good idea. I might recommend reserving the last bit of the second byte to be a flag such as DIRTY_BITMAP_EXTRA_FLAGS that indicates the presence of
Re: [Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
On 16.02.2015 21:18, John Snow wrote: On 02/16/2015 07:06 AM, Vladimir Sementsov-Ogievskiy wrote: On 13.02.2015 23:22, John Snow wrote: On 02/13/2015 03:19 AM, Vladimir Sementsov-Ogievskiy wrote: On 11.02.2015 00:33, John Snow wrote: So in summary: using device names is probably fine for now, as it matches the current use case of bitmaps as well as drive migration; but using node names may give us more power and precision later. I talked to Max about it, and he is leaning towards using device names for now and switching to node names if we decide we want that power. (...I wonder if we could use a flag, for now, that says we're including DEVICE names. Later, we could add a flag that says we're using NODE names and add an option to toggle as the usage case sees fit.) Are you confused yet? :D O, thanks for the explanation). Are we really need this flag? As Markus wrote, nodes and devices are sharing namespaces.. We can use bdrv_lookup_bs(name, name, errp).. what 'name' are you using here, though? It looked to me like in your backup routine we got a list of BDS entries and get the name *from* the BDS, so we still have to think about how we want to /get/ the name. Also, we can, for example, send bitmaps as follows: if node has name - send bitmap with this name if node is root, but hasn't name - send it with blk name otherwise - don't send the bitmap The node a bitmap is attached to should always have a name -- it would not be possible via the existing interface to attach it to a node without a name. I *think* the root node should always have a name, but I am actually less sure of that. Hmm.. No? bitmap is attached using bdrv_lookup_bs(name, name, errp), which can find device with this name. qemu option -drive file=...,id=disk creates blk named 'disk' and attached node with no name. -- Best regards, Vladimir
[Qemu-devel] RFC RFC
Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org/Contribute/SubmitAPatch -- Best regards, Vladimir
[Qemu-devel] [PATCH RFC v3 01/14] qmp: add query-block-dirty-bitmap
Adds qmp and hmp commands to query/info dirty bitmap. This is needed only for
testing.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 41
blockdev.c| 24 +
hmp-commands.hx | 2 ++
hmp.c | 32 ++
hmp.h | 1 +
include/block/block.h | 2 ++
monitor.c | 7 +
qapi/block-core.json | 75 +++
qmp-commands.hx | 5
9 files changed, 189 insertions(+)
diff --git a/block.c b/block.c
index f3a6dd4..e4547d7 100644
--- a/block.c
+++ b/block.c
@@ -5574,6 +5574,47 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
return list;
}
+BlockDirtyBitmapInfo *bdrv_query_dirty_bitmap(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap)
+{
+BlockDirtyBitmapInfo *info = g_new0(BlockDirtyBitmapInfo, 1);
+BlockDirtyRegionList **plist = &info->dirty_regions;
+uint64_t begin = 0, end = 0, size = bitmap->size;
+HBitmap *hb = bitmap->bitmap;
+
+info->dirty_count = bdrv_get_dirty_count(bitmap);
+info->granularity = bdrv_dirty_bitmap_granularity(bitmap);
+info->size = bitmap->size;
+info->name = g_strdup(bitmap->name);
+info->disabled = bitmap->disabled;
+info->dirty_regions = NULL;
+
+for (; begin < size; ++begin) {
+BlockDirtyRegion *region;
+BlockDirtyRegionList *entry;
+
+if (!hbitmap_get(hb, begin)) {
+continue;
+}
+
+for (end = begin + 1; end < size && hbitmap_get(hb, end); ++end) {
+;
+}
+
+region = g_new0(BlockDirtyRegion, 1);
+entry = g_new0(BlockDirtyRegionList, 1);
+region->start = begin;
+region->count = end - begin;
+entry->value = region;
+*plist = entry;
+plist = &entry->next;
+
+begin = end;
+}
+
+return info;
+}
+
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t
sector)
{
if (bitmap) {
diff --git a/blockdev.c b/blockdev.c
index 13068c7..ad08ea0 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2323,6 +2323,30 @@ void qmp_block_dirty_bitmap_clear(const char *node,
const char *name,
aio_context_release(aio_context);
}
+BlockDirtyBitmapInfo *qmp_query_block_dirty_bitmap(const char *node,
+ const char *name,
+ Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+BlockDirtyBitmapInfo *ret = NULL;
+
+bitmap = block_dirty_bitmap_lookup(node, name, &bs, errp);
+if (!bitmap) {
+return NULL;
+}
+
+aio_context = bdrv_get_aio_context(bs);
+aio_context_acquire(aio_context);
+
+ret = bdrv_query_dirty_bitmap(bs, bitmap);
+
+aio_context_release(aio_context);
+
+return ret;
+}
+
int do_drive_del(Monitor *mon, const QDict *qdict, QObject **ret_data)
{
const char *id = qdict_get_str(qdict, "id");
diff --git a/hmp-commands.hx b/hmp-commands.hx
index e37bc8b..8c5e580 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1778,6 +1778,8 @@ show roms
show the TPM device
@item info memory-devices
show the memory devices
+@item info dirty-bitmap
+show dirty bitmap details and content
@end table
ETEXI
diff --git a/hmp.c b/hmp.c
index 015499f..2f7ce87 100644
--- a/hmp.c
+++ b/hmp.c
@@ -788,6 +788,38 @@ void hmp_info_tpm(Monitor *mon, const QDict *qdict)
qapi_free_TPMInfoList(info_list);
}
+void hmp_info_block_dirty_bitmap(Monitor *mon, const QDict *qdict)
+{
+const char *device = qdict_get_str(qdict, "device");
+const char *name = qdict_get_str(qdict, "bitmap");
+BlockDirtyRegionList *region;
+
+BlockDirtyBitmapInfo *info =
+qmp_query_block_dirty_bitmap(device, name, NULL);
+
+if (!info) {
+monitor_printf(mon, "bitmap '%s' for device '%s' doesn't exis\n",
+ name, device);
+return;
+}
+
+monitor_printf(mon, "bitmap '%s'\n",
+ info->name ? info->name : "no name");
+monitor_printf(mon, "disabled: %s\n", info->disabled ? "true" : "false");
+monitor_printf(mon, "size: %" PRId64 "\n", info->size);
+monitor_printf(mon, "granularity: %" PRIu64 "\n", info->granularity);
+monitor_printf(mon, "dirty regions begin\n");
+
+for (region = info->dirty_regions; region; region = region->next) {
+monitor_printf(mon, "%" PRIu64 ": %" PRIu64 "\n",
+ region->value->start, region->value->count);
+}
+
[Qemu-devel] [PATCH RFC v3 08/14] migration: add migration/block-dirty-bitmap.c
Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
root nodes and non-root named nodes are migrated.
If destination qemu is already containing a dirty bitmap with the same name
as a migrated bitmap (for the same node), than, if their granularities are
the same the migration will be done, otherwise the error will be generated.
If destination qemu doesn't contain such bitmap it will be created.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/block.h | 1 +
migration/Makefile.objs| 2 +-
migration/block-dirty-bitmap.c | 708 +
vl.c | 1 +
4 files changed, 711 insertions(+), 1 deletion(-)
create mode 100644 migration/block-dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..128612d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o block-dirty-bitmap.o
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
new file mode 100644
index 000..084ba22
--- /dev/null
+++ b/migration/block-dirty-bitmap.c
@@ -0,0 +1,708 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
+ * root nodes and non-root named nodes are migrated.
+ *
+ * If destination qemu is already containing a dirty bitmap with the same name
+ * as a migrated bitmap (for the same node), than, if their granularities are
+ * the same the migration will be done, otherwise the error will be generated.
+ *
+ * If destination qemu doesn't contain such bitmap it will be created.
+ *
+ * format of migration:
+ *
+ * # Header (shared for different chunk types)
+ * 1 byte: flags
+ * [ 1 byte: node name size ] \ flags & DEVICE_NAME
+ * [ n bytes: node name ] /
+ * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME
+ * [ n bytes: bitmap name ] /
+ *
+ * # Start of bitmap migration (flags & START)
+ * header
+ * be64: granularity
+ *
+ * # Complete of bitmap migration (flags & COMPLETE)
+ * header
+ * 1 byte: bitmap enabled flag
+ *
+ * # Data chunk of bitmap migration
+ * header
+ * be64: start sector
+ * be32: number of sectors
+ * [ be64: buffer size ] \ ! (flags & ZEROES)
+ * [ n bytes: buffer] /
+ *
+ * The last chunk in stream should contain flags & EOS. The chunk may skip
+ * device and/or bitmap names, assuming them to be the same with the previous
+ * chunk.
+ *
+ *
+ * This file is derived from migration/block.c
+ *
+ * Author:
+ * Vladimir Sementsov-Ogievskiy
+ *
+ * original copyright message:
+ * =
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ * =
+ */
+
+#include "block/block.h"
+#include "block/block_int.h"
+#include "sysemu/block-backend.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include
+
+#define CHUNK_SIZE (1 << 20)
+
+/* Flags occupy from one to four bytes. In all but one the 7-th (EXTRA_FLAGS)
+ * bit should be set. */
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_ZEROES0x02
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_START 0x10
+#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20
+#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40
+
+#define DIRTY_BITMAP_MIG_EXTRA_FLAGS0x80
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080
+
+#define DEBUG_DIRTY_BITMAP_MIGRATION
+
+#ifdef DEBUG_DIRTY_BITMAP_MIGRATION
+#define DPRINTF(fmt, ...) \
+do { printf("dirty_migration: " fmt, ## __VA_ARGS__); } while (0)
+#else
+#define DPRINTF(fmt, ...) \
+do { } while (0)
+#endif
+
+typedef struct DirtyBitmapMigBitmapState {
+/* Written during setup phase
[Qemu-devel] [PATCH RFC v3 02/14] hbitmap: serialization
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bitmap migration, restoring
the bitmap in several steps is available. To save performance, every
step writes only the last level of the bitmap. All other levels are
restored by hbitmap_deserialize_finish() as a last step of restoring.
So, HBitmap is inconsistent while restoring.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/qemu/hbitmap.h | 59 +++
util/hbitmap.c | 96 ++
2 files changed, 155 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index c19c1cb..10ce05b 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -129,6 +129,65 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t
count);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_data_size:
+ * @hb: HBitmap to operate on.
+ * @count: Number of bits
+ *
+ * Return amount of bytes hbitmap_serialize_part needs
+ */
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count);
+
+/**
+ * hbitmap_serialize_part
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to store serialized bitmap.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
+ * independently of endianness and size of HBitmap level array elements
+ */
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Retores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_serialize_part.
+ *
+ * ! The bitmap becomes inconsistent after this operation.
+ * hbitmap_serialize_finish should be called before using the bitmap after
+ * data restoring.
+ */
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_zeroes
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Same as hbitmap_serialize_part, but fills the bitmap with zeroes.
+ */
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
+ * layers are restored here.
+ */
+void hbitmap_deserialize_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 962ff29..1a736e7 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -362,6 +362,102 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size, gran;
+
+if (count == 0) {
+return 0;
+}
+
+gran = 1ll << hb->granularity;
+size = (((gran + count - 2) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *out = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+out[i] = (BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
+}
+}
+
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *in = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+hb->levels[HBITMAP_LEVELS - 1][i] =
+(BITS_PER_LONG == 32 ? le32_to_cpu(in[i]) : le64_to_cpu(in[i]));
+}
+}
+
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t co
[Qemu-devel] [PATCH RFC v3 13/14] iotests: add dirty bitmap migration test
[the same test as 117, but not using qmp: query-block-dirty-bitmap.
only one test from {117, 118} will be in the next patch set version]
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b with dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/118 | 87 ++
tests/qemu-iotests/118.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 93 insertions(+)
create mode 100755 tests/qemu-iotests/118
create mode 100644 tests/qemu-iotests/118.out
diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
new file mode 100755
index 000..a85ea56
--- /dev/null
+++ b/tests/qemu-iotests/118
@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+#
+# Tests for dirty bitmaps migration.
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk_a = os.path.join(iotests.test_dir, 'disk_a')
+disk_b = os.path.join(iotests.test_dir, 'disk_b')
+fifo = os.path.join(iotests.test_dir, 'fifo')
+
+size = 0x4000 # 1G
+sector_size = 512
+granularity = 0x1
+regions = [
+{ 'start': 0, 'count': 0x10 },
+{ 'start': 0x1000, 'count': 0x2 },
+{ 'start': 0x3999, 'count': 0x1 }
+]
+
+regions_in_sectors = [
+{ key: val / sector_size for (key, val) in el.items() } for el in regions]
+
+class TestDirtyBitmapMigration(iotests.QMPTestCase):
+
+def setUp(self):
+os.mkfifo(fifo)
+qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size))
+qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size))
+self.vm_a = iotests.VM().add_drive(disk_a)
+self.vm_b = iotests.VM().add_drive(disk_b)
+self.vm_b.add_incoming_migration("exec: cat " + fifo)
+self.vm_a.launch()
+self.vm_b.launch()
+
+def tearDown(self):
+self.vm_a.shutdown()
+self.vm_b.shutdown()
+os.remove(disk_a)
+os.remove(disk_b)
+os.remove(fifo)
+
+def test_migration(self):
+result = self.vm_a.qmp('block-dirty-bitmap-add', node='drive0',
+ name='bitmap', granularity=granularity)
+self.assert_qmp(result, 'return', {});
+
+for r in regions:
+ self.vm_a.hmp_qemu_io('drive0',
+'write %d %d' % (r['start'], r['count']))
+
+result = self.vm_a.qmp('query-block');
+md5 = result['return'][0]['dirty-bitmaps'][0]['md5']
+
+result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'dirty-bitmaps',
+ 'state': True}])
+self.assert_qmp(result, 'return', {})
+
+result = self.vm_a.qmp('migrate', uri='exec:cat>' + fifo)
+while self.vm_a.qmp('query-migrate')['return']['status'] !=
'completed':
+ time.sleep(1)
+
+result = self.vm_b.qmp('query-block');
+self.assert_qmp(result, 'return[0]/dirty-bitmaps[0]/md5', md5);
+
+
+if __name__ == '__main__':
+iotests.main()
diff --git a/tests/qemu-iotests/118.out b/tests/qemu-iotests/118.out
new file mode 100644
index 000..ae1213e
--- /dev/null
+++ b/tests/qemu-iotests/118.out
@@ -0,0 +1,5 @@
+.
+--
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6ad5b55..1fd1983 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -119,3 +119,4 @@
114 rw auto quick
116 rw auto quick
117 rw auto quick
+118 rw auto quick
--
1.9.1
[Qemu-devel] [PATCH RFC v3 09/14] iotests: maintain several vms in test
The only problem with it is the same qmp socket name (which is vm._monitor_path) for all vms. And because of this second vm couldn't be lauched (vm.launch() fails because of socket is already in use). This patch adds a number of vm into vm._monitor_path Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index cf5faac..fa756b4 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -77,16 +77,18 @@ def create_image(name, size): class VM(object): '''A QEMU VM''' +nb_vms = 0 def __init__(self): -self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d' % os.getpid()) -self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d' % os.getpid()) +self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d.%d' % (os.getpid(), VM.nb_vms)) +self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d.%d' % (os.getpid(), VM.nb_vms)) self._args = qemu_args + ['-chardev', 'socket,id=mon,path=' + self._monitor_path, '-mon', 'chardev=mon,mode=control', '-qtest', 'stdio', '-machine', 'accel=qtest', '-display', 'none', '-vga', 'none'] self._num_drives = 0 +VM.nb_vms += 1 # This can be used to add an unused monitor instance. def add_monitor_telnet(self, ip, port): -- 1.9.1
[Qemu-devel] [PATCH RFC v3 03/14] block: BdrvDirtyBitmap serialization interface
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 36
include/block/block.h | 13 +
2 files changed, 49 insertions(+)
diff --git a/block.c b/block.c
index e4547d7..e95a5ae 100644
--- a/block.c
+++ b/block.c
@@ -5674,6 +5674,42 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
}
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+return bitmap->name;
+}
+
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count)
+{
+return hbitmap_data_size(bitmap->bitmap, count);
+}
+
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+hbitmap_serialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count)
+{
+hbitmap_deserialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count)
+{
+hbitmap_deserialize_zeroes(bitmap->bitmap, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
+{
+hbitmap_deserialize_finish(bitmap->bitmap);
+}
+
static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index c9d96a6..c6a928d 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -474,6 +474,19 @@ void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, struct
HBitmapIter *hbi);
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count);
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count);
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count);
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v3 06/14] block: add bdrv_next_dirty_bitmap()
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide
access to private dirty bitmaps list.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 10 ++
include/block/block.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/block.c b/block.c
index aaa08b8..4cca55d 100644
--- a/block.c
+++ b/block.c
@@ -5649,6 +5649,16 @@ BlockDirtyBitmapInfo
*bdrv_query_dirty_bitmap(BlockDriverState *bs,
return info;
}
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap)
+{
+if (bitmap == NULL) {
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+return QLIST_NEXT(bitmap, list);
+}
+
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t
sector)
{
if (bitmap) {
diff --git a/include/block/block.h b/include/block/block.h
index f2c62f6..d11d956 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -487,6 +487,8 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap
*bitmap,
void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap);
HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
uint64_t granularity);
--
1.9.1
[Qemu-devel] [PATCH RFC v3 00/14] Dirty bitmaps migration
v3: based on v13 of "block: incremental backup series" by John Snow. changes from v2: removed patch for adding dirty parameter (migration capablities used instead). 0001: printf's dropped, qapi used 0002: part0 -> zeroes 0003: part0 -> zeroes 0005: dirty_dirty -> meta add comments about meta bitmap 0006: the format is changed, nodes used instead of devices. other patches are new. rfc: there are two tests. They are the same but using different interfaces: md5 checksum of the bitmap last layer in query-block or separate query-block-dirty-bitmap with dirty bitmap regions. The second form is more appropriate for debugging, the first is more appropriate for simple regression control. Which should go to upstream? v2: 1. bug-fixes, that are already in upstream, and renaming of function bdrv_reset_dirty_bitmap (which is already in Snow's series) are dropped 2. bitmap store/restore: the concept renamed to serialization, added function hbitmap_deserialize_part0, to not transfer zero blocks 3. migration dirty parameter: added description comment 4. Other patches are new. v2.rfc: Actually, in this version of the series I'm trying not use migration/block.c at all. Instead a separate migration unit is added in the new file migration/dirty-bitmap.c. Now bitmaps are migrated like blocks in block migration, they have their "dirty-dirty" bitmaps, for tracking set/unset changes during migration. The advantages are: - no complications of migration/block.c - separate dirty-dirty bitmaps provide handling of "unset's" - more effective meta-data/data ratio - no tiny bitmap-blocks. v1: These patches provide dirty bitmap migration feature. Only named dirty bitmaps are to be migrated. Migration is made as a part of block migration in block-migration.c. Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate command. If "blk" and "inc" parameters are false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John Snow. Vladimir Sementsov-Ogievskiy (14): qmp: add query-block-dirty-bitmap hbitmap: serialization block: BdrvDirtyBitmap serialization interface block: tiny refactoring: minimize hbitmap_(set/reset) usage block: add meta bitmaps block: add bdrv_next_dirty_bitmap() qapi: add dirty-bitmaps migration capability migration: add migration/block-dirty-bitmap.c iotests: maintain several vms in test iotests: add add_incoming_migration to VM class iotests: add dirty bitmap migration test qapi: add md5 checksum of last dirty bitmap level to query-block iotests: add dirty bitmap migration test migration/qemu-file: make functions qemu_(get/put)_string public block.c| 135 - blockdev.c | 24 ++ hmp-commands.hx| 2 + hmp.c | 32 ++ hmp.h | 1 + include/block/block.h | 22 ++ include/migration/block.h | 1 + include/migration/migration.h | 1 + include/migration/qemu-file.h | 17 ++ include/qemu/hbitmap.h | 67 migration/Makefile.objs| 2 +- migration/block-dirty-bitmap.c | 673 + migration/migration.c | 9 + migration/qemu-file.c | 18 ++ monitor.c | 7 + qapi-schema.json | 5 +- qapi/block-core.json | 79 - qmp-commands.hx| 5 + tests/qemu-iotests/117 | 88 ++ tests/qemu-iotests/117.out | 5 + tests/qemu-iotests/118 | 87 ++ tests/qemu-iotests/118.out | 5 + tests/qemu-iotests/group | 2 + tests/qemu-iotests/iotests.py | 12 +- util/hbitmap.c | 104 +++ vl.c | 1 + 26 files changed, 1395 insertions(+), 9 deletions(-) create mode 100644 migration/block-dirty-bitmap.c create mode 100755 tests/qemu-iotests/117 create mode 100644 tests/qemu-iotests/117.out create mode 100755 tests/qemu-iotests/118 create mode 100644 tests/qemu-iotests/118.out -- 1.9.1
Re: [Qemu-devel] RFC RFC
On 18.02.2015 17:01, Eric Blake wrote: On 02/18/2015 01:10 AM, Vladimir Sementsov-Ogievskiy wrote: Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org/Contribute/SubmitAPatch Peter's response was great, but this page is a wiki. Would you like to add the information yourself, now that you have it? Hm.. I have no account on the wiki and registration is disabled. -- Best regards, Vladimir
[Qemu-devel] [PATCH RFC v3 04/14] block: tiny refactoring: minimize hbitmap_(set/reset) usage
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/block.c b/block.c
index e95a5ae..a127fd2 100644
--- a/block.c
+++ b/block.c
@@ -5670,8 +5670,7 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
-hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+bdrv_reset_dirty_bitmap(bitmap, 0, bitmap->size);
}
const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
@@ -5718,7 +5717,7 @@ static void bdrv_set_dirty(BlockDriverState *bs, int64_t
cur_sector,
if (!bdrv_dirty_bitmap_enabled(bitmap)) {
continue;
}
-hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+bdrv_set_dirty_bitmap(bitmap, cur_sector, nr_sectors);
}
}
@@ -5730,7 +5729,7 @@ static void bdrv_reset_dirty(BlockDriverState *bs,
int64_t cur_sector,
if (!bdrv_dirty_bitmap_enabled(bitmap)) {
continue;
}
-hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+bdrv_reset_dirty_bitmap(bitmap, cur_sector, nr_sectors);
}
}
--
1.9.1
[Qemu-devel] [PATCH RFC v3 07/14] qapi: add dirty-bitmaps migration capability
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/migration.h | 1 +
migration/migration.c | 9 +
qapi-schema.json | 5 -
3 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/include/migration/migration.h b/include/migration/migration.h
index f37348b..719875d 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -145,6 +145,7 @@ void migrate_del_blocker(Error *reason);
bool migrate_rdma_pin_all(void);
bool migrate_zero_blocks(void);
+bool migrate_dirty_bitmaps(void);
bool migrate_auto_converge(void);
diff --git a/migration/migration.c b/migration/migration.c
index b3adbc6..68e7478 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -567,6 +567,15 @@ bool migrate_zero_blocks(void)
return s->enabled_capabilities[MIGRATION_CAPABILITY_ZERO_BLOCKS];
}
+bool migrate_dirty_bitmaps(void)
+{
+MigrationState *s;
+
+s = migrate_get_current();
+
+return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS];
+}
+
int migrate_use_xbzrle(void)
{
MigrationState *s;
diff --git a/qapi-schema.json b/qapi-schema.json
index 0c31203..70b54ab 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -494,10 +494,13 @@
# @auto-converge: If enabled, QEMU will automatically throttle down the guest
# to speed up convergence of RAM migration. (since 1.6)
#
+# @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. (since
2.3)
+#
# Since: 1.2
##
{ 'enum': 'MigrationCapability',
- 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks'] }
+ 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks',
+ 'dirty-bitmaps'] }
##
# @MigrationCapabilityStatus
--
1.9.1
[Qemu-devel] [PATCH RFC v3 12/14] qapi: add md5 checksum of last dirty bitmap level to query-block
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c| 1 +
include/qemu/hbitmap.h | 8
qapi/block-core.json | 4 +++-
util/hbitmap.c | 8
4 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/block.c b/block.c
index 4cca55d..9532ccc 100644
--- a/block.c
+++ b/block.c
@@ -5600,6 +5600,7 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
info->name = g_strdup(bm->name);
info->disabled = bm->disabled;
info->frozen = bdrv_dirty_bitmap_frozen(bm);
+info->md5 = hbitmap_md5(bm->bitmap);
entry->value = info;
*plist = entry;
plist = &entry->next;
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 10ce05b..2fb748a 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -188,6 +188,14 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t
start, uint64_t count);
void hbitmap_deserialize_finish(HBitmap *hb);
/**
+ * hbitmap_md5:
+ * @bitmap: HBitmap to operate on.
+ *
+ * Returns md5 checksum of the last level.
+ */
+char *hbitmap_md5(const HBitmap *bitmap);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 25dea80..2028d37 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -336,11 +336,13 @@
#
# @frozen: whether the dirty bitmap is frozen (Since 2.3)
#
+# @md5: md5 checksum of the last bitmap level (since 2.3)
+#
# Since: 1.3
##
{ 'type': 'BlockDirtyInfo',
'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
- 'disabled': 'bool', 'frozen': 'bool'} }
+ 'disabled': 'bool', 'frozen': 'bool', 'md5': 'str'} }
##
# @BlockInfo:
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 1a736e7..8063dce 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -523,3 +523,11 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b)
return true;
}
+
+char *hbitmap_md5(const HBitmap *bitmap)
+{
+uint64_t size =
+MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+const guchar *data = (const guchar *)bitmap->levels[HBITMAP_LEVELS - 1];
+return g_compute_checksum_for_data(G_CHECKSUM_MD5, data, size);
+}
--
1.9.1
[Qemu-devel] [PATCH RFC v3 10/14] iotests: add add_incoming_migration to VM class
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/iotests.py | 6 ++
1 file changed, 6 insertions(+)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fa756b4..75640b2 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -111,6 +111,12 @@ class VM(object):
self._num_drives += 1
return self
+def add_incoming_migration(self, desc):
+'''Add an incoming migration to the VM'''
+self._args.append('-incoming')
+self._args.append(desc)
+return self
+
def pause_drive(self, drive, event=None):
'''Pause drive r/w operations'''
if not event:
--
1.9.1
[Qemu-devel] [PATCH RFC v3 14/14] migration/qemu-file: make functions qemu_(get/put)_string public
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/qemu-file.h | 17 +
migration/block-dirty-bitmap.c | 35 ---
migration/qemu-file.c | 18 ++
3 files changed, 35 insertions(+), 35 deletions(-)
diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index a923cec..178ae63 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -310,4 +310,21 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t
*pv)
{
qemu_get_be64s(f, (uint64_t *)pv);
}
+
+/* read name from qemu file:
+ * format:
+ * 1 byte : len = name length (<256)
+ * len bytes : name without last zero byte
+ *
+ * name should point to the buffer >= 256 bytes length
+ */
+char *qemu_get_string(QEMUFile *f, char *name);
+
+/* write name to qemu file:
+ * format:
+ * same as for qemu_get_string
+ *
+ * maximum name length is 255
+ */
+void qemu_put_string(QEMUFile *f, const char *name);
#endif
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 084ba22..8ebf7e5 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -161,41 +161,6 @@ static void qemu_put_flags(QEMUFile *f, uint32_t flags)
qemu_put_be32(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_32);
}
-/* read name from qemu file:
- * format:
- * 1 byte : len = name length (<256)
- * len bytes : name without last zero byte
- *
- * name should point to the buffer >= 256 bytes length
- */
-static char *qemu_get_string(QEMUFile *f, char *name)
-{
-int len = qemu_get_byte(f);
-qemu_get_buffer(f, (uint8_t *)name, len);
-name[len] = '\0';
-
-DPRINTF("get name: %d %s\n", len, name);
-
-return name;
-}
-
-/* write name to qemu file:
- * format:
- * same as for qemu_get_string
- *
- * maximum name length is 255
- */
-static void qemu_put_string(QEMUFile *f, const char *name)
-{
-int len = strlen(name);
-
-DPRINTF("put name: %d %s\n", len, name);
-
-assert(len < 256);
-qemu_put_byte(f, len);
-qemu_put_buffer(f, (const uint8_t *)name, len);
-}
-
static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
uint32_t additional_flags)
{
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index e66e557..5439f84 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -545,3 +545,21 @@ uint64_t qemu_get_be64(QEMUFile *f)
v |= qemu_get_be32(f);
return v;
}
+
+char *qemu_get_string(QEMUFile *f, char *name)
+{
+int len = qemu_get_byte(f);
+qemu_get_buffer(f, (uint8_t *)name, len);
+name[len] = '\0';
+
+return name;
+}
+
+void qemu_put_string(QEMUFile *f, const char *name)
+{
+int len = strlen(name);
+
+assert(len < 256);
+qemu_put_byte(f, len);
+qemu_put_buffer(f, (const uint8_t *)name, len);
+}
--
1.9.1
[Qemu-devel] [PATCH RFC v3 05/14] block: add meta bitmaps
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks
changes (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of block dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 40
include/block/block.h | 5 +
2 files changed, 45 insertions(+)
diff --git a/block.c b/block.c
index a127fd2..aaa08b8 100644
--- a/block.c
+++ b/block.c
@@ -58,9 +58,15 @@
* (3) successor is set: frozen mode.
* A frozen bitmap cannot be renamed, deleted, anonymized, cleared, set,
* or enabled. A frozen bitmap can only abdicate() or reclaim().
+ *
+ * Meta bitmap:
+ * Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes
+ * (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of
+ * block dirty bitmaps.
*/
struct BdrvDirtyBitmap {
HBitmap *bitmap;/* Dirty sector bitmap implementation */
+HBitmap *meta_bitmap; /* Meta bitmap */
BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of sectors) */
@@ -5398,6 +5404,31 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
bitmap->name = NULL;
}
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity)
+{
+uint64_t sector_granularity;
+
+assert((granularity & (granularity - 1)) == 0);
+
+granularity *= 8 * bdrv_dirty_bitmap_granularity(bitmap);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+
+bitmap->meta_bitmap =
+hbitmap_alloc(bitmap->size, ffsll(sector_granularity) - 1);
+
+return bitmap->meta_bitmap;
+}
+
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap)
+{
+if (bitmap->meta_bitmap) {
+hbitmap_free(bitmap->meta_bitmap);
+bitmap->meta_bitmap = NULL;
+}
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
uint32_t granularity,
const char *name,
@@ -5532,6 +5563,9 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
assert(!bdrv_dirty_bitmap_frozen(bm));
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
+if (bitmap->meta_bitmap) {
+hbitmap_free(bitmap->meta_bitmap);
+}
g_free(bitmap->name);
g_free(bitmap);
return;
@@ -5659,6 +5693,9 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->meta_bitmap) {
+hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+}
}
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
@@ -5666,6 +5703,9 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->meta_bitmap) {
+hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+}
}
void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
diff --git a/include/block/block.h b/include/block/block.h
index c6a928d..f2c62f6 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,6 +4,7 @@
#include "block/aio.h"
#include "qemu-common.h"
#include "qemu/option.h"
+#include "qemu/hbitmap.h"
#include "block/coroutine.h"
#include "block/accounting.h"
#include "qapi/qmp/qobject.h"
@@ -487,6 +488,10 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap
*bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity);
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v3 11/14] iotests: add dirty bitmap migration test
The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do
several writes to corresponding device and then migrate vm_a to vm_b
with dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/117 | 88 ++
tests/qemu-iotests/117.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 94 insertions(+)
create mode 100755 tests/qemu-iotests/117
create mode 100644 tests/qemu-iotests/117.out
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
new file mode 100755
index 000..61538cf
--- /dev/null
+++ b/tests/qemu-iotests/117
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+#
+# Tests for dirty bitmaps migration.
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk_a = os.path.join(iotests.test_dir, 'disk_a')
+disk_b = os.path.join(iotests.test_dir, 'disk_b')
+fifo = os.path.join(iotests.test_dir, 'fifo')
+
+size = 0x4000 # 1G
+sector_size = 512
+granularity = 0x1
+regions = [
+{ 'start': 0, 'count': 0x10 },
+{ 'start': 0x1000, 'count': 0x2 },
+{ 'start': 0x3999, 'count': 0x1 }
+]
+
+regions_in_sectors = [
+{ key: val / sector_size for (key, val) in el.items() } for el in regions]
+
+class TestDirtyBitmapMigration(iotests.QMPTestCase):
+
+def setUp(self):
+os.mkfifo(fifo)
+qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size))
+qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size))
+self.vm_a = iotests.VM().add_drive(disk_a)
+self.vm_b = iotests.VM().add_drive(disk_b)
+self.vm_b.add_incoming_migration("exec: cat " + fifo)
+self.vm_a.launch()
+self.vm_b.launch()
+
+def tearDown(self):
+self.vm_a.shutdown()
+self.vm_b.shutdown()
+os.remove(disk_a)
+os.remove(disk_b)
+os.remove(fifo)
+
+def test_migration(self):
+result = self.vm_a.qmp('block-dirty-bitmap-add', node='drive0',
+ name='bitmap', granularity=granularity)
+self.assert_qmp(result, 'return', {});
+
+for r in regions:
+ self.vm_a.hmp_qemu_io('drive0',
+'write %d %d' % (r['start'], r['count']))
+
+result = self.vm_a.qmp('query-block-dirty-bitmap', node='drive0',
+ name='bitmap')
+self.assert_qmp(result, 'return/dirty-regions', regions_in_sectors)
+
+result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'dirty-bitmaps',
+ 'state': True}])
+self.assert_qmp(result, 'return', {})
+result = self.vm_a.qmp('migrate', uri='exec:cat>' + fifo)
+while self.vm_a.qmp('query-migrate')['return']['status'] !=
'completed':
+ time.sleep(1)
+
+result = self.vm_b.qmp('query-block-dirty-bitmap', node='drive0',
+ name='bitmap')
+self.assert_qmp(result, 'return/dirty-regions', regions_in_sectors);
+
+
+if __name__ == '__main__':
+iotests.main()
diff --git a/tests/qemu-iotests/117.out b/tests/qemu-iotests/117.out
new file mode 100644
index 000..ae1213e
--- /dev/null
+++ b/tests/qemu-iotests/117.out
@@ -0,0 +1,5 @@
+.
+--
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index b4ddf1b..6ad5b55 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -118,3 +118,4 @@
113 rw auto quick
114 rw auto quick
116 rw auto quick
+117 rw auto quick
--
1.9.1
Re: [Qemu-devel] RFC RFC
On 18.02.2015 17:01, Eric Blake wrote: On 02/18/2015 01:10 AM, Vladimir Sementsov-Ogievskiy wrote: Hi Can anybody explain what the reasons, politics and consequences of adding RFC into patch header in this mailing list? I think, it is not worth to add this information to http://wiki.qemu.org/Contribute/SubmitAPatch Peter's response was great, but this page is a wiki. Would you like to add the information yourself, now that you have it? done. -- Best regards, Vladimir
Re: [Qemu-devel] [PATCH RFC v3 05/14] block: add meta bitmaps
On 19.02.2015 02:45, John Snow wrote:
On 02/18/2015 09:00 AM, Vladimir Sementsov-Ogievskiy wrote:
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks
changes (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of block dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 40
include/block/block.h | 5 +
2 files changed, 45 insertions(+)
diff --git a/block.c b/block.c
index a127fd2..aaa08b8 100644
--- a/block.c
+++ b/block.c
@@ -58,9 +58,15 @@
* (3) successor is set: frozen mode.
* A frozen bitmap cannot be renamed, deleted, anonymized,
cleared, set,
* or enabled. A frozen bitmap can only abdicate() or reclaim().
+ *
+ * Meta bitmap:
+ * Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It
tracks changes
+ * (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of
+ * block dirty bitmaps.
*/
struct BdrvDirtyBitmap {
HBitmap *bitmap;/* Dirty sector bitmap
implementation */
+HBitmap *meta_bitmap; /* Meta bitmap */
BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen
status */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of
sectors) */
@@ -5398,6 +5404,31 @@ void
bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
bitmap->name = NULL;
}
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity)
+{
+uint64_t sector_granularity;
+
+assert((granularity & (granularity - 1)) == 0);
+
+granularity *= 8 * bdrv_dirty_bitmap_granularity(bitmap);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+
The maths here could use a comment, I think.
the "granularity" field here actually describes the desired
serialization buffer size; e.g. CHUNK_SIZE (1 << 20) or 1 MiB. This
parameter should be renamed to explain what it's actually for.
Something like "chunk_size" and a comment explaining that it is in bytes.
...
That said, let's talk about the default chunk size you're using in
correlation with this function.
a CHUNK_SIZE of 1MiB here is going to lead us to, if we have a bitmap
with the default granularity of 128 sectors\64KiB bytes, a granularity
for the meta_bitmap of one billion sectors (1 << 30) or 512GiB.
That's going to be bigger than most drives entirely, which will
generally lead us to only using a single "chunk" per drive. Which
means we won't really get a lot of mileage out of the bulk/dirty
phases most of the time.
It's wild to think about that the first 1,000,000,000 sectors or
512,000,000,000 bytes will all be represented by the first single bit
in this bitmap. If a single hair on the drive changes, we resend the
_entire_ bitmap, possibly over and over again. Will we ever make
progress? Should we investigate a smaller chunk size?
Here's some quick mappings of chunk size (bytes) to effective
meta_bitmap byte granularities, assuming the meta_bitmap is tracking a
bitmap with the default granularity of 64KiB:
(1 << 20) 1MiB -- 512GiB // This is too high of a granularity
(1 << 17) 128KiB -- 64GiB
(1 << 15) 32KiB -- 16GiB
(1 << 11) 2KiB -- 1GiB
(1 << 10) 1KiB -- 512MiB
(1 << 9) 512B -- 256MiB
(1 << 8) 256B -- 128MiB
(1 << 5) 32 B -- 16MiB // This is too small of a chunk size.
(1 << 1) 1 B -- 1MiB
We want to make the chunk sends efficient, but we also want to make
sure that the dirty phase doesn't resend more data than it needs to,
so we need to strike a balance here, no?
I think arguments could be made for most granularities between 128MiB
through 1GiB. Anything outside of that is too lopsided, IMO.
What are your thoughts on this?
Ok, interesting thing to discuss.
My thoughts:
* the chunk size for block-migration is 1mb, than the bitmap (64kb
granularity) for this chunk is 16bit=2bytes long. It's an intuitive
reason for choosing the chunk size about 2 bytes. But in this case the
data/metadata ratio is very bad (about 20bytes for the header of the
chunk). So, taking the nearest value with adequate ratio gives (IMHO)
'1kb -- 512mb': 20b/1k ~ 2%. Or 512b => 4%.
* for ndb+mirror migration scheme the default chunk is 64kb instead of
1mb. So the bitmap is more smaller. But the same reason of data/metadata
ratio leads to 1kb chunk for dirty bitmap migration.
So, what about default to 1kb and additional parameter for migration
(migration capabilities) to give the user a possibility of chose?
* Yes, in most of user cases the bitmap (64kb granularity) will be small
(< 1mb). In these cases, I think, it would be better to send
Re: [Qemu-devel] [PATCH RFC v3 08/14] migration: add migration/block-dirty-bitmap.c
On 19.02.2015 02:47, John Snow wrote:
On 02/18/2015 09:00 AM, Vladimir Sementsov-Ogievskiy wrote:
Live migration of dirty bitmaps. Only named dirty bitmaps, associated
with
root nodes and non-root named nodes are migrated.
If destination qemu is already containing a dirty bitmap with the
same name
as a migrated bitmap (for the same node), than, if their
granularities are
the same the migration will be done, otherwise the error will be
generated.
If destination qemu doesn't contain such bitmap it will be created.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/block.h | 1 +
migration/Makefile.objs| 2 +-
migration/block-dirty-bitmap.c | 708
+
vl.c | 1 +
4 files changed, 711 insertions(+), 1 deletion(-)
create mode 100644 migration/block-dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..128612d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o block-dirty-bitmap.o
diff --git a/migration/block-dirty-bitmap.c
b/migration/block-dirty-bitmap.c
new file mode 100644
index 000..084ba22
--- /dev/null
+++ b/migration/block-dirty-bitmap.c
@@ -0,0 +1,708 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * Live migration of dirty bitmaps. Only named dirty bitmaps,
associated with
+ * root nodes and non-root named nodes are migrated.
+ *
+ * If destination qemu is already containing a dirty bitmap with the
same name
+ * as a migrated bitmap (for the same node), than, if their
granularities are
+ * the same the migration will be done, otherwise the error will be
generated.
+ *
+ * If destination qemu doesn't contain such bitmap it will be created.
+ *
+ * format of migration:
+ *
+ * # Header (shared for different chunk types)
+ * 1 byte: flags
1, 2, or 4 bytes.
+ * [ 1 byte: node name size ] \ flags & DEVICE_NAME
+ * [ n bytes: node name ] /
+ * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME
+ * [ n bytes: bitmap name ] /
+ *
+ * # Start of bitmap migration (flags & START)
+ * header
+ * be64: granularity
+ *
+ * # Complete of bitmap migration (flags & COMPLETE)
+ * header
+ * 1 byte: bitmap enabled flag
+ *
+ * # Data chunk of bitmap migration
+ * header
+ * be64: start sector
+ * be32: number of sectors
+ * [ be64: buffer size ] \ ! (flags & ZEROES)
+ * [ n bytes: buffer] /
+ *
+ * The last chunk in stream should contain flags & EOS. The chunk
may skip
+ * device and/or bitmap names, assuming them to be the same with the
previous
+ * chunk.
+ *
+ *
+ * This file is derived from migration/block.c
+ *
+ * Author:
+ * Vladimir Sementsov-Ogievskiy
+ *
+ * original copyright message:
+ *
=
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ *
=
+ */
+
+#include "block/block.h"
+#include "block/block_int.h"
+#include "sysemu/block-backend.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include
+
+#define CHUNK_SIZE (1 << 20)
+
+/* Flags occupy from one to four bytes. In all but one the 7-th
(EXTRA_FLAGS)
+ * bit should be set. */
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_ZEROES0x02
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_START 0x10
+#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20
+#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40
+
+#define DIRTY_BITMAP_MIG_EXTRA_FLAGS0x80
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080
+
+#define DEBUG_DIRTY_BITMAP_MIGRATION
+
+#ifdef DEBUG_DIRTY_BITMAP_MIGRATION
+#define DPRINTF(fmt, ...) \
+do { printf("dirty_migration: " fmt, ## __VA_AR
[Qemu-devel] [PATCH RFC v4 09/13] iotests: add add_incoming_migration to VM class
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/iotests.py | 6 ++
1 file changed, 6 insertions(+)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index fa756b4..75640b2 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -111,6 +111,12 @@ class VM(object):
self._num_drives += 1
return self
+def add_incoming_migration(self, desc):
+'''Add an incoming migration to the VM'''
+self._args.append('-incoming')
+self._args.append(desc)
+return self
+
def pause_drive(self, drive, event=None):
'''Pause drive r/w operations'''
if not event:
--
1.9.1
[Qemu-devel] [PATCH RFC v4 03/13] block: tiny refactoring: minimize hbitmap_(set/reset) usage
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/block.c b/block.c
index e95a5ae..a127fd2 100644
--- a/block.c
+++ b/block.c
@@ -5670,8 +5670,7 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
-assert(bdrv_dirty_bitmap_enabled(bitmap));
-hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+bdrv_reset_dirty_bitmap(bitmap, 0, bitmap->size);
}
const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
@@ -5718,7 +5717,7 @@ static void bdrv_set_dirty(BlockDriverState *bs, int64_t
cur_sector,
if (!bdrv_dirty_bitmap_enabled(bitmap)) {
continue;
}
-hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+bdrv_set_dirty_bitmap(bitmap, cur_sector, nr_sectors);
}
}
@@ -5730,7 +5729,7 @@ static void bdrv_reset_dirty(BlockDriverState *bs,
int64_t cur_sector,
if (!bdrv_dirty_bitmap_enabled(bitmap)) {
continue;
}
-hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+bdrv_reset_dirty_bitmap(bitmap, cur_sector, nr_sectors);
}
}
--
1.9.1
[Qemu-devel] [PATCH RFC v4 08/13] iotests: maintain several vms in test
The only problem with it is the same qmp socket name (which is vm._monitor_path) for all vms. And because of this second vm couldn't be lauched (vm.launch() fails because of socket is already in use). This patch adds a number of vm into vm._monitor_path Reviewed-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index cf5faac..fa756b4 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -77,16 +77,18 @@ def create_image(name, size): class VM(object): '''A QEMU VM''' +nb_vms = 0 def __init__(self): -self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d' % os.getpid()) -self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d' % os.getpid()) +self._monitor_path = os.path.join(test_dir, 'qemu-mon.%d.%d' % (os.getpid(), VM.nb_vms)) +self._qemu_log_path = os.path.join(test_dir, 'qemu-log.%d.%d' % (os.getpid(), VM.nb_vms)) self._args = qemu_args + ['-chardev', 'socket,id=mon,path=' + self._monitor_path, '-mon', 'chardev=mon,mode=control', '-qtest', 'stdio', '-machine', 'accel=qtest', '-display', 'none', '-vga', 'none'] self._num_drives = 0 +VM.nb_vms += 1 # This can be used to add an unused monitor instance. def add_monitor_telnet(self, ip, port): -- 1.9.1
[Qemu-devel] [PATCH RFC v4 05/13] block: add bdrv_next_dirty_bitmap()
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide
access to private dirty bitmaps list.
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 10 ++
include/block/block.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/block.c b/block.c
index e77be2e..7b6251c 100644
--- a/block.c
+++ b/block.c
@@ -5653,6 +5653,16 @@ BlockDirtyBitmapInfo
*bdrv_query_dirty_bitmap(BlockDriverState *bs,
return info;
}
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap)
+{
+if (bitmap == NULL) {
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+return QLIST_NEXT(bitmap, list);
+}
+
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t
sector)
{
if (bitmap) {
diff --git a/include/block/block.h b/include/block/block.h
index 00e34b5..ae08d51 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -487,6 +487,8 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap
*bitmap,
void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap);
/* chunk size here is number of bytes of the @bitmap data per one bit of the
* meta bitmap being created */
--
1.9.1
[Qemu-devel] [PATCH RFC v4 11/13] qapi: add md5 checksum of last dirty bitmap level to query-block
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c| 1 +
include/qemu/hbitmap.h | 8
qapi/block-core.json | 4 +++-
util/hbitmap.c | 8
4 files changed, 20 insertions(+), 1 deletion(-)
diff --git a/block.c b/block.c
index 7b6251c..96bcc05 100644
--- a/block.c
+++ b/block.c
@@ -5604,6 +5604,7 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
info->name = g_strdup(bm->name);
info->disabled = bm->disabled;
info->frozen = bdrv_dirty_bitmap_frozen(bm);
+info->md5 = hbitmap_md5(bm->bitmap);
entry->value = info;
*plist = entry;
plist = &entry->next;
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 10ce05b..2fb748a 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -188,6 +188,14 @@ void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t
start, uint64_t count);
void hbitmap_deserialize_finish(HBitmap *hb);
/**
+ * hbitmap_md5:
+ * @bitmap: HBitmap to operate on.
+ *
+ * Returns md5 checksum of the last level.
+ */
+char *hbitmap_md5(const HBitmap *bitmap);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 25dea80..2028d37 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -336,11 +336,13 @@
#
# @frozen: whether the dirty bitmap is frozen (Since 2.3)
#
+# @md5: md5 checksum of the last bitmap level (since 2.3)
+#
# Since: 1.3
##
{ 'type': 'BlockDirtyInfo',
'data': {'*name': 'str', 'count': 'int', 'granularity': 'uint32',
- 'disabled': 'bool', 'frozen': 'bool'} }
+ 'disabled': 'bool', 'frozen': 'bool', 'md5': 'str'} }
##
# @BlockInfo:
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 1a123da..8de94eb 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -525,3 +525,11 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b)
return true;
}
+
+char *hbitmap_md5(const HBitmap *bitmap)
+{
+uint64_t size =
+MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+const guchar *data = (const guchar *)bitmap->levels[HBITMAP_LEVELS - 1];
+return g_compute_checksum_for_data(G_CHECKSUM_MD5, data, size);
+}
--
1.9.1
[Qemu-devel] [PATCH RFC v4 04/13] block: add meta bitmaps
Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks
changes (set/unset) of this BdrvDirtyBitmap. It is needed for live
migration of block dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 44
include/block/block.h | 7 +++
2 files changed, 51 insertions(+)
diff --git a/block.c b/block.c
index a127fd2..e77be2e 100644
--- a/block.c
+++ b/block.c
@@ -58,9 +58,15 @@
* (3) successor is set: frozen mode.
* A frozen bitmap cannot be renamed, deleted, anonymized, cleared, set,
* or enabled. A frozen bitmap can only abdicate() or reclaim().
+ *
+ * Meta bitmap:
+ * Meta bitmap is a 'dirty bitmap' for the BdrvDirtyBitmap. It tracks changes
+ * (set/unset) of this BdrvDirtyBitmap. It is needed for live migration of
+ * block dirty bitmaps.
*/
struct BdrvDirtyBitmap {
HBitmap *bitmap;/* Dirty sector bitmap implementation */
+HBitmap *meta_bitmap; /* Meta bitmap */
BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of sectors) */
@@ -5398,6 +5404,35 @@ void bdrv_dirty_bitmap_make_anon(BdrvDirtyBitmap *bitmap)
bitmap->name = NULL;
}
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+ uint64_t chunk_size)
+{
+uint64_t sector_granularity;
+
+assert((chunk_size & (chunk_size - 1)) == 0);
+
+/* one chunk is corresponding to one bit of the meta bitmap, and each bit
+ * of the chunk is corresponding to 'bdrv_dirty_bitmap_granularity(bitmap)'
+ * bytes of the node */
+sector_granularity =
+(chunk_size * 8 * bdrv_dirty_bitmap_granularity(bitmap))
+>> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+
+bitmap->meta_bitmap =
+hbitmap_alloc(bitmap->size, ffsll(sector_granularity) - 1);
+
+return bitmap->meta_bitmap;
+}
+
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap)
+{
+if (bitmap->meta_bitmap) {
+hbitmap_free(bitmap->meta_bitmap);
+bitmap->meta_bitmap = NULL;
+}
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
uint32_t granularity,
const char *name,
@@ -5532,6 +5567,9 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
assert(!bdrv_dirty_bitmap_frozen(bm));
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
+if (bitmap->meta_bitmap) {
+hbitmap_free(bitmap->meta_bitmap);
+}
g_free(bitmap->name);
g_free(bitmap);
return;
@@ -5659,6 +5697,9 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->meta_bitmap) {
+hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+}
}
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
@@ -5666,6 +5707,9 @@ void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
{
assert(bdrv_dirty_bitmap_enabled(bitmap));
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->meta_bitmap) {
+hbitmap_set(bitmap->meta_bitmap, cur_sector, nr_sectors);
+}
}
void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
diff --git a/include/block/block.h b/include/block/block.h
index c6a928d..00e34b5 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,6 +4,7 @@
#include "block/aio.h"
#include "qemu-common.h"
#include "qemu/option.h"
+#include "qemu/hbitmap.h"
#include "block/coroutine.h"
#include "block/accounting.h"
#include "qapi/qmp/qobject.h"
@@ -487,6 +488,12 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap
*bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+/* chunk size here is number of bytes of the @bitmap data per one bit of the
+ * meta bitmap being created */
+HBitmap *bdrv_create_meta_bitmap(BdrvDirtyBitmap *bitmap,
+ uint64_t granularity);
+void bdrv_release_meta_bitmap(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v4 06/13] qapi: add dirty-bitmaps migration capability
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/migration.h | 1 +
migration/migration.c | 9 +
qapi-schema.json | 5 -
3 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/include/migration/migration.h b/include/migration/migration.h
index f37348b..719875d 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -145,6 +145,7 @@ void migrate_del_blocker(Error *reason);
bool migrate_rdma_pin_all(void);
bool migrate_zero_blocks(void);
+bool migrate_dirty_bitmaps(void);
bool migrate_auto_converge(void);
diff --git a/migration/migration.c b/migration/migration.c
index b3adbc6..68e7478 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -567,6 +567,15 @@ bool migrate_zero_blocks(void)
return s->enabled_capabilities[MIGRATION_CAPABILITY_ZERO_BLOCKS];
}
+bool migrate_dirty_bitmaps(void)
+{
+MigrationState *s;
+
+s = migrate_get_current();
+
+return s->enabled_capabilities[MIGRATION_CAPABILITY_DIRTY_BITMAPS];
+}
+
int migrate_use_xbzrle(void)
{
MigrationState *s;
diff --git a/qapi-schema.json b/qapi-schema.json
index 0c31203..70b54ab 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -494,10 +494,13 @@
# @auto-converge: If enabled, QEMU will automatically throttle down the guest
# to speed up convergence of RAM migration. (since 1.6)
#
+# @dirty-bitmaps: If enabled, QEMU will migrate named dirty bitmaps. (since
2.3)
+#
# Since: 1.2
##
{ 'enum': 'MigrationCapability',
- 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks'] }
+ 'data': ['xbzrle', 'rdma-pin-all', 'auto-converge', 'zero-blocks',
+ 'dirty-bitmaps'] }
##
# @MigrationCapabilityStatus
--
1.9.1
[Qemu-devel] [PATCH RFC v4 10/13] iotests: add event_wait to VM class
Signed-off-by: John Snow Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/iotests.py | 7 +++ 1 file changed, 7 insertions(+) diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py index 75640b2..294b158 100644 --- a/tests/qemu-iotests/iotests.py +++ b/tests/qemu-iotests/iotests.py @@ -203,6 +203,13 @@ class VM(object): self._qmp.clear_events() return events +def event_wait(self, name='BLOCK_JOB_COMPLETED', maxwait=10): +for _ in range(maxwait): +for event in self.get_qmp_events(wait=True): +if event['event'] == name: +return event +return None + index_re = re.compile(r'([^\[]+)\[([^\]]+)\]') class QMPTestCase(unittest.TestCase): -- 1.9.1
[Qemu-devel] [PATCH RFC v4 13/13] migration/qemu-file: make functions qemu_(get/put)_string public
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/qemu-file.h | 17 +
migration/block-dirty-bitmap.c | 35 ---
migration/qemu-file.c | 18 ++
3 files changed, 35 insertions(+), 35 deletions(-)
diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index a923cec..178ae63 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -310,4 +310,21 @@ static inline void qemu_get_sbe64s(QEMUFile *f, int64_t
*pv)
{
qemu_get_be64s(f, (uint64_t *)pv);
}
+
+/* read name from qemu file:
+ * format:
+ * 1 byte : len = name length (<256)
+ * len bytes : name without last zero byte
+ *
+ * name should point to the buffer >= 256 bytes length
+ */
+char *qemu_get_string(QEMUFile *f, char *name);
+
+/* write name to qemu file:
+ * format:
+ * same as for qemu_get_string
+ *
+ * maximum name length is 255
+ */
+void qemu_put_string(QEMUFile *f, const char *name);
#endif
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
index 2d3ba23..3c610af 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
@@ -164,41 +164,6 @@ static void qemu_put_flags(QEMUFile *f, uint32_t flags)
qemu_put_be32(f, flags | DIRTY_BITMAP_MIG_FLAGS_SIZE_32);
}
-/* read name from qemu file:
- * format:
- * 1 byte : len = name length (<256)
- * len bytes : name without last zero byte
- *
- * name should point to the buffer >= 256 bytes length
- */
-static char *qemu_get_string(QEMUFile *f, char *name)
-{
-int len = qemu_get_byte(f);
-qemu_get_buffer(f, (uint8_t *)name, len);
-name[len] = '\0';
-
-DPRINTF("get name: %d %s\n", len, name);
-
-return name;
-}
-
-/* write name to qemu file:
- * format:
- * same as for qemu_get_string
- *
- * maximum name length is 255
- */
-static void qemu_put_string(QEMUFile *f, const char *name)
-{
-int len = strlen(name);
-
-DPRINTF("put name: %d %s\n", len, name);
-
-assert(len < 256);
-qemu_put_byte(f, len);
-qemu_put_buffer(f, (const uint8_t *)name, len);
-}
-
static void send_bitmap_header(QEMUFile *f, DirtyBitmapMigBitmapState *dbms,
uint32_t additional_flags)
{
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index e66e557..5439f84 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -545,3 +545,21 @@ uint64_t qemu_get_be64(QEMUFile *f)
v |= qemu_get_be32(f);
return v;
}
+
+char *qemu_get_string(QEMUFile *f, char *name)
+{
+int len = qemu_get_byte(f);
+qemu_get_buffer(f, (uint8_t *)name, len);
+name[len] = '\0';
+
+return name;
+}
+
+void qemu_put_string(QEMUFile *f, const char *name)
+{
+int len = strlen(name);
+
+assert(len < 256);
+qemu_put_byte(f, len);
+qemu_put_buffer(f, (const uint8_t *)name, len);
+}
--
1.9.1
[Qemu-devel] [PATCH RFC v4 02/13] block: BdrvDirtyBitmap serialization interface
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 36
include/block/block.h | 13 +
2 files changed, 49 insertions(+)
diff --git a/block.c b/block.c
index e4547d7..e95a5ae 100644
--- a/block.c
+++ b/block.c
@@ -5674,6 +5674,42 @@ void bdrv_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap)
hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
}
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+return bitmap->name;
+}
+
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count)
+{
+return hbitmap_data_size(bitmap->bitmap, count);
+}
+
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+hbitmap_serialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count)
+{
+hbitmap_deserialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count)
+{
+hbitmap_deserialize_zeroes(bitmap->bitmap, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
+{
+hbitmap_deserialize_finish(bitmap->bitmap);
+}
+
static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index c9d96a6..c6a928d 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -474,6 +474,19 @@ void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, struct
HBitmapIter *hbi);
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count);
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count);
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count);
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v4 01/13] hbitmap: serialization
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bitmap migration, restoring
the bitmap in several steps is available. To save performance, every
step writes only the last level of the bitmap. All other levels are
restored by hbitmap_deserialize_finish() as a last step of restoring.
So, HBitmap is inconsistent while restoring.
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/qemu/hbitmap.h | 59 ++
util/hbitmap.c | 98 ++
2 files changed, 157 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index c19c1cb..10ce05b 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -129,6 +129,65 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t
count);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_data_size:
+ * @hb: HBitmap to operate on.
+ * @count: Number of bits
+ *
+ * Return amount of bytes hbitmap_serialize_part needs
+ */
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count);
+
+/**
+ * hbitmap_serialize_part
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to store serialized bitmap.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
+ * independently of endianness and size of HBitmap level array elements
+ */
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Retores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_serialize_part.
+ *
+ * ! The bitmap becomes inconsistent after this operation.
+ * hbitmap_serialize_finish should be called before using the bitmap after
+ * data restoring.
+ */
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_zeroes
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Same as hbitmap_serialize_part, but fills the bitmap with zeroes.
+ */
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
+ * layers are restored here.
+ */
+void hbitmap_deserialize_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 962ff29..1a123da 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -362,6 +362,104 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size, gran;
+
+if (count == 0) {
+return 0;
+}
+
+gran = 1ll << hb->granularity;
+size = (((gran + count - 2) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *out = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+out[i - start] =
+(BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
+}
+}
+
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *in = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+hb->levels[HBITMAP_LEVELS - 1][i] =
+(BITS_PER_LONG == 32 ? le32_to_cpu(in[i - start]) :
+ le64_to_
[Qemu-devel] [PATCH RFC v4 00/13] Dirty bitmaps migration
These patches provide dirty bitmap migration feature. Only named dirty bitmaps are to be migrated. Migration may be enabled using migration capabilities. v4 significant changes: 0001: tiny bugfix: out[i] -> out[i-start], same for 'in' 0007: set chunk size to 1kb, disable live iteration for migrating data < 1mb size. tests: only one with md5 sum is here. used function event_wait by John Snow. (I hope, you don't mind me just adding this function with your 'Signed-off-by') rfc: This patch set is based on v13 of "block: incremental backup series" by John Snow, which are not pushed yet. v3: based on v13 of "block: incremental backup series" by John Snow. changes from v2: removed patch for adding dirty parameter (migration capablities used instead). 0001: printf's dropped, qapi used 0002: part0 -> zeroes 0003: part0 -> zeroes 0005: dirty_dirty -> meta add comments about meta bitmap 0006: the format is changed, nodes used instead of devices. other patches are new. rfc: there are two tests. They are the same but using different interfaces: md5 checksum of the bitmap last layer in query-block or separate query-block-dirty-bitmap with dirty bitmap regions. The second form is more appropriate for debugging, the first is more appropriate for simple regression control. Which should go to upstream? v2: 1. bug-fixes, that are already in upstream, and renaming of function bdrv_reset_dirty_bitmap (which is already in Snow's series) are dropped 2. bitmap store/restore: the concept renamed to serialization, added function hbitmap_deserialize_part0, to not transfer zero blocks 3. migration dirty parameter: added description comment 4. Other patches are new. v2.rfc: Actually, in this version of the series I'm trying not use migration/block.c at all. Instead a separate migration unit is added in the new file migration/dirty-bitmap.c. Now bitmaps are migrated like blocks in block migration, they have their "dirty-dirty" bitmaps, for tracking set/unset changes during migration. The advantages are: - no complications of migration/block.c - separate dirty-dirty bitmaps provide handling of "unset's" - more effective meta-data/data ratio - no tiny bitmap-blocks. v1: These patches provide dirty bitmap migration feature. Only named dirty bitmaps are to be migrated. Migration is made as a part of block migration in block-migration.c. Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate command. If "blk" and "inc" parameters are false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John Snow. *** BLURB HERE *** Vladimir Sementsov-Ogievskiy (13): hbitmap: serialization block: BdrvDirtyBitmap serialization interface block: tiny refactoring: minimize hbitmap_(set/reset) usage block: add meta bitmaps block: add bdrv_next_dirty_bitmap() qapi: add dirty-bitmaps migration capability migration: add migration/block-dirty-bitmap.c iotests: maintain several vms in test iotests: add add_incoming_migration to VM class iotests: add event_wait to VM class qapi: add md5 checksum of last dirty bitmap level to query-block iotests: add dirty bitmap migration test migration/qemu-file: make functions qemu_(get/put)_string public block.c| 98 +- include/block/block.h | 22 ++ include/migration/block.h | 1 + include/migration/migration.h | 1 + include/migration/qemu-file.h | 17 + include/qemu/hbitmap.h | 67 migration/Makefile.objs| 2 +- migration/block-dirty-bitmap.c | 693 + migration/migration.c | 9 + migration/qemu-file.c | 18 ++ qapi-schema.json | 5 +- qapi/block-core.json | 4 +- tests/qemu-iotests/117 | 84 + tests/qemu-iotests/117.out | 5 + tests/qemu-iotests/group | 1 + tests/qemu-iotests/iotests.py | 19 +- util/hbitmap.c | 106 +++ vl.c | 1 + 18 files changed, 1144 insertions(+), 9 deletions(-) create mode 100644 migration/block-dirty-bitmap.c create mode 100755 tests/qemu-iotests/117 create mode 100644 tests/qemu-iotests/117.out -- 1.9.1
[Qemu-devel] [PATCH RFC v4 12/13] iotests: add dirty bitmap migration test
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b with dirty bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/117 | 84 ++
tests/qemu-iotests/117.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 90 insertions(+)
create mode 100755 tests/qemu-iotests/117
create mode 100644 tests/qemu-iotests/117.out
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
new file mode 100755
index 000..9fab5d0
--- /dev/null
+++ b/tests/qemu-iotests/117
@@ -0,0 +1,84 @@
+#!/usr/bin/env python
+#
+# Tests for dirty bitmaps migration.
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk_a = os.path.join(iotests.test_dir, 'disk_a')
+disk_b = os.path.join(iotests.test_dir, 'disk_b')
+fifo = os.path.join(iotests.test_dir, 'fifo')
+
+size = 0x4000 # 1G
+sector_size = 512
+granularity = 0x1
+regions = [
+{ 'start': 0, 'count': 0x10 },
+{ 'start': 0x1000, 'count': 0x2 },
+{ 'start': 0x3999, 'count': 0x1 }
+]
+
+class TestDirtyBitmapMigration(iotests.QMPTestCase):
+
+def setUp(self):
+os.mkfifo(fifo)
+qemu_img('create', '-f', iotests.imgfmt, disk_a, str(size))
+qemu_img('create', '-f', iotests.imgfmt, disk_b, str(size))
+self.vm_a = iotests.VM().add_drive(disk_a)
+self.vm_b = iotests.VM().add_drive(disk_b)
+self.vm_b.add_incoming_migration("exec: cat " + fifo)
+self.vm_a.launch()
+self.vm_b.launch()
+
+def tearDown(self):
+self.vm_a.shutdown()
+self.vm_b.shutdown()
+os.remove(disk_a)
+os.remove(disk_b)
+os.remove(fifo)
+
+def test_migration(self):
+result = self.vm_a.qmp('block-dirty-bitmap-add', node='drive0',
+ name='bitmap', granularity=granularity)
+self.assert_qmp(result, 'return', {});
+
+for r in regions:
+ self.vm_a.hmp_qemu_io('drive0',
+'write %d %d' % (r['start'], r['count']))
+
+result = self.vm_a.qmp('query-block');
+md5 = result['return'][0]['dirty-bitmaps'][0]['md5']
+
+result = self.vm_a.qmp('migrate-set-capabilities',
+ capabilities=[{'capability': 'dirty-bitmaps',
+ 'state': True}])
+self.assert_qmp(result, 'return', {})
+
+result = self.vm_a.qmp('migrate', uri='exec:cat>' + fifo)
+self.assertIsNotNone(self.vm_a.event_wait("STOP"))
+self.assertIsNotNone(self.vm_b.event_wait("RESUME"))
+
+result = self.vm_b.qmp('query-block');
+self.assert_qmp(result, 'return[0]/dirty-bitmaps[0]/md5', md5);
+
+
+if __name__ == '__main__':
+iotests.main()
diff --git a/tests/qemu-iotests/117.out b/tests/qemu-iotests/117.out
new file mode 100644
index 000..ae1213e
--- /dev/null
+++ b/tests/qemu-iotests/117.out
@@ -0,0 +1,5 @@
+.
+--
+Ran 1 tests
+
+OK
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index b4ddf1b..6ad5b55 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -118,3 +118,4 @@
113 rw auto quick
114 rw auto quick
116 rw auto quick
+117 rw auto quick
--
1.9.1
[Qemu-devel] [PATCH RFC v4 07/13] migration: add migration/block-dirty-bitmap.c
Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
root nodes and non-root named nodes are migrated.
If destination qemu is already containing a dirty bitmap with the same name
as a migrated bitmap (for the same node), than, if their granularities are
the same the migration will be done, otherwise the error will be generated.
If destination qemu doesn't contain such bitmap it will be created.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/block.h | 1 +
migration/Makefile.objs| 2 +-
migration/block-dirty-bitmap.c | 728 +
vl.c | 1 +
4 files changed, 731 insertions(+), 1 deletion(-)
create mode 100644 migration/block-dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..128612d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o block-dirty-bitmap.o
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
new file mode 100644
index 000..2d3ba23
--- /dev/null
+++ b/migration/block-dirty-bitmap.c
@@ -0,0 +1,728 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * Live migration of dirty bitmaps. Only named dirty bitmaps, associated with
+ * root nodes and non-root named nodes are migrated. Live iteration is disabled
+ * for small data amount (see MIN_LIVE_SIZE).
+ *
+ * If destination qemu is already containing a dirty bitmap with the same name
+ * as a migrated bitmap (for the same node), than, if their granularities are
+ * the same the migration will be done, otherwise the error will be generated.
+ *
+ * If destination qemu doesn't contain such bitmap it will be created.
+ *
+ * format of migration:
+ *
+ * # Header (shared for different chunk types)
+ * 1, 2 or 4 bytes: flags (see qemu_{put,put}_flags)
+ * [ 1 byte: node name size ] \ flags & DEVICE_NAME
+ * [ n bytes: node name ] /
+ * [ 1 byte: bitmap name size ] \ flags & BITMAP_NAME
+ * [ n bytes: bitmap name ] /
+ *
+ * # Start of bitmap migration (flags & START)
+ * header
+ * be64: granularity
+ *
+ * # Complete of bitmap migration (flags & COMPLETE)
+ * header
+ * 1 byte: bitmap enabled flag
+ *
+ * # Data chunk of bitmap migration
+ * header
+ * be64: start sector
+ * be32: number of sectors
+ * [ be64: buffer size ] \ ! (flags & ZEROES)
+ * [ n bytes: buffer] /
+ *
+ * The last chunk in stream should contain flags & EOS. The chunk may skip
+ * device and/or bitmap names, assuming them to be the same with the previous
+ * chunk.
+ *
+ *
+ * This file is derived from migration/block.c
+ *
+ * Author:
+ * Vladimir Sementsov-Ogievskiy
+ *
+ * original copyright message:
+ * =
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ * =
+ */
+
+#include "block/block.h"
+#include "block/block_int.h"
+#include "sysemu/block-backend.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include
+
+#define CHUNK_SIZE (1 << 10)
+#define MIN_LIVE_SIZE (1 << 20)
+
+/* Flags occupy from one to four bytes. In all but one the 7-th (EXTRA_FLAGS)
+ * bit should be set. */
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_ZEROES0x02
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x04
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_START 0x10
+#define DIRTY_BITMAP_MIG_FLAG_COMPLETE 0x20
+#define DIRTY_BITMAP_MIG_FLAG_BITS 0x40
+
+#define DIRTY_BITMAP_MIG_EXTRA_FLAGS0x80
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_16 0x8000
+#define DIRTY_BITMAP_MIG_FLAGS_SIZE_32 0x8080
+
+#define DEBUG_DIRTY_BITMAP_MIGRATION 1
+
+#define DPRINTF(fmt, args...) \
+do { \
+if (DEBUG_DIRTY_BITMAP_MIGRATION) { \
+printf("DMIG %s:%d", __func__, __LINE__);
[Qemu-devel] [PATCH RFC v2 3/8] block: BdrvDirtyBitmap serialization interface
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 36
include/block/block.h | 12
2 files changed, 48 insertions(+)
diff --git a/block.c b/block.c
index 6d3f0b2..9860fc1 100644
--- a/block.c
+++ b/block.c
@@ -5552,6 +5552,42 @@ void bdrv_clear_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
}
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+return bitmap->name;
+}
+
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count)
+{
+return hbitmap_data_size(bitmap->bitmap, count);
+}
+
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+hbitmap_serialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count)
+{
+hbitmap_deserialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part0(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count)
+{
+hbitmap_deserialize_part0(bitmap->bitmap, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
+{
+hbitmap_deserialize_finish(bitmap->bitmap);
+}
+
static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index 6cf067f..0890cd2 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -460,6 +460,18 @@ void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t
offset);
int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+uint64_t bdrv_dirty_bitmap_data_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t count);
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+uint8_t *buf, uint64_t start,
+uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part0(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count);
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v2 4/8] block: add dirty-dirty bitmaps
A dirty-dirty bitmap is a dirty bitmap for BdrvDirtyBitmap. It tracks
set/unset changes of BdrvDirtyBitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 44
include/block/block.h | 5 +
2 files changed, 49 insertions(+)
diff --git a/block.c b/block.c
index 9860fc1..8ab724b 100644
--- a/block.c
+++ b/block.c
@@ -53,6 +53,7 @@
struct BdrvDirtyBitmap {
HBitmap *bitmap;
+HBitmap *dirty_dirty_bitmap;
BdrvDirtyBitmap *originator;
int64_t size;
int64_t granularity;
@@ -5373,6 +5374,30 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
return originator;
}
+HBitmap *bdrv_create_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity)
+{
+uint64_t sector_granularity;
+
+assert((granularity & (granularity - 1)) == 0);
+
+granularity *= 8 * bitmap->granularity;
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
+
+bitmap->dirty_dirty_bitmap =
+hbitmap_alloc(bitmap->size, ffsll(sector_granularity) - 1);
+
+return bitmap->dirty_dirty_bitmap;
+}
+
+void bdrv_release_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_free(bitmap->dirty_dirty_bitmap);
+bitmap->dirty_dirty_bitmap = NULL;
+}
+}
void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
@@ -5447,6 +5472,9 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
if (bm == bitmap) {
QLIST_REMOVE(bitmap, list);
hbitmap_free(bitmap->bitmap);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_free(bitmap->dirty_dirty_bitmap);
+}
g_free(bitmap->name);
g_free(bitmap);
return;
@@ -5534,6 +5562,10 @@ void bdrv_set_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
{
if (bitmap->enabled) {
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector, nr_sectors);
+}
}
}
@@ -5541,6 +5573,9 @@ void bdrv_reset_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors)
{
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector, nr_sectors);
+}
}
/**
@@ -5550,6 +5585,9 @@ void bdrv_reset_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
void bdrv_clear_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
{
hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, 0, bitmap->size);
+}
}
const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
@@ -5597,6 +5635,9 @@ static void bdrv_set_dirty(BlockDriverState *bs, int64_t
cur_sector,
continue;
}
hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector, nr_sectors);
+}
}
}
@@ -5606,6 +5647,9 @@ static void bdrv_reset_dirty(BlockDriverState *bs,
int64_t cur_sector,
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+if (bitmap->dirty_dirty_bitmap) {
+hbitmap_set(bitmap->dirty_dirty_bitmap, cur_sector, nr_sectors);
+}
}
}
diff --git a/include/block/block.h b/include/block/block.h
index 0890cd2..648b0a9 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -4,6 +4,7 @@
#include "block/aio.h"
#include "qemu-common.h"
#include "qemu/option.h"
+#include "qemu/hbitmap.h"
#include "block/coroutine.h"
#include "block/accounting.h"
#include "qapi/qmp/qobject.h"
@@ -473,6 +474,10 @@ void bdrv_dirty_bitmap_deserialize_part0(BdrvDirtyBitmap
*bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+HBitmap *bdrv_create_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap,
+uint64_t granularity);
+void bdrv_release_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH RFC v2 6/8] block: add bdrv_next_dirty_bitmap()
Like bdrv_next() - bdrv_next_dirty_bitmap() is a function to provide
access to private dirty bitmaps list.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 10 ++
include/block/block.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/block.c b/block.c
index 15fc621..9e59c2e 100644
--- a/block.c
+++ b/block.c
@@ -5514,6 +5514,16 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs)
return list;
}
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap)
+{
+if (bitmap == NULL) {
+return QLIST_FIRST(&bs->dirty_bitmaps);
+}
+
+return QLIST_NEXT(bitmap, list);
+}
+
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t
sector)
{
if (bitmap) {
diff --git a/include/block/block.h b/include/block/block.h
index 7b49d98..34d0259 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -474,6 +474,8 @@ void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap
*bitmap,
void bdrv_dirty_bitmap_deserialize_part0(BdrvDirtyBitmap *bitmap,
uint64_t start, uint64_t count);
void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap *bdrv_next_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap);
HBitmap *bdrv_create_dirty_dirty_bitmap(BdrvDirtyBitmap *bitmap,
uint64_t granularity);
--
1.9.1
[Qemu-devel] [PATCH RFC v2 7/8] migration: add dirty parameter
Add dirty parameter to qmp-migrate command. If this parameter is true,
migration/block.c will migrate dirty bitmaps. This parameter can be used
without "blk" parameter to migrate only dirty bitmaps, skipping block
migration.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
hmp-commands.hx | 12 +++-
hmp.c | 4 +++-
include/migration/migration.h | 1 +
migration/migration.c | 4 +++-
qapi-schema.json | 7 ++-
qmp-commands.hx | 5 -
savevm.c | 3 ++-
7 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index a9be506..c16f93c 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -902,23 +902,25 @@ ETEXI
{
.name = "migrate",
-.args_type = "detach:-d,blk:-b,inc:-i,uri:s",
-.params = "[-d] [-b] [-i] uri",
+.args_type = "detach:-d,blk:-b,inc:-i,dirty:-D,uri:s",
+.params = "[-d] [-b] [-i] [-D] uri",
.help = "migrate to URI (using -d to not wait for completion)"
"\n\t\t\t -b for migration without shared storage with"
" full copy of disk\n\t\t\t -i for migration without "
- "shared storage with incremental copy of disk "
- "(base image shared between src and destination)",
+ "shared storage with incremental copy of disk\n\t\t\t"
+ " -D for migration of named dirty bitmaps as well\n\t\t\t"
+ " (base image shared between src and destination)",
.mhandler.cmd = hmp_migrate,
},
STEXI
-@item migrate [-d] [-b] [-i] @var{uri}
+@item migrate [-d] [-b] [-i] [-D] @var{uri}
@findex migrate
Migrate to @var{uri} (using -d to not wait for completion).
-b for migration with full copy of disk
-i for migration with incremental copy of disk (base image is shared)
+ -D for migration of named dirty bitmaps
ETEXI
{
diff --git a/hmp.c b/hmp.c
index a269145..0b89ee8 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1347,10 +1347,12 @@ void hmp_migrate(Monitor *mon, const QDict *qdict)
int detach = qdict_get_try_bool(qdict, "detach", 0);
int blk = qdict_get_try_bool(qdict, "blk", 0);
int inc = qdict_get_try_bool(qdict, "inc", 0);
+int dirty = qdict_get_try_bool(qdict, "dirty", 0);
const char *uri = qdict_get_str(qdict, "uri");
Error *err = NULL;
-qmp_migrate(uri, !!blk, blk, !!inc, inc, false, false, &err);
+qmp_migrate(uri, !!blk, blk, !!inc, inc, !!dirty, dirty,
+false, false, &err);
if (err) {
monitor_printf(mon, "migrate: %s\n", error_get_pretty(err));
error_free(err);
diff --git a/include/migration/migration.h b/include/migration/migration.h
index 3cb5ba8..48d71d3 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -37,6 +37,7 @@
struct MigrationParams {
bool blk;
bool shared;
+bool dirty;
};
typedef struct MigrationState MigrationState;
diff --git a/migration/migration.c b/migration/migration.c
index c49a05a..e7bb7f3 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -404,7 +404,8 @@ void migrate_del_blocker(Error *reason)
}
void qmp_migrate(const char *uri, bool has_blk, bool blk,
- bool has_inc, bool inc, bool has_detach, bool detach,
+ bool has_inc, bool inc, bool has_dirty, bool dirty,
+ bool has_detach, bool detach,
Error **errp)
{
Error *local_err = NULL;
@@ -414,6 +415,7 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk,
params.blk = has_blk && blk;
params.shared = has_inc && inc;
+params.dirty = has_dirty && dirty;
if (s->state == MIG_STATE_ACTIVE || s->state == MIG_STATE_SETUP ||
s->state == MIG_STATE_CANCELLING) {
diff --git a/qapi-schema.json b/qapi-schema.json
index 1475f69..1d10d6b 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1656,12 +1656,17 @@
# @detach: this argument exists only for compatibility reasons and
# is ignored by QEMU
#
+# @dirty: #optional do dirty-bitmaps migration (can be used with or without
+# @blk parameter)
+# (since 2.3)
+#
# Returns: nothing on success
#
# Since: 0.14.0
##
{ 'command': 'migrate',
- 'data': {'uri': 'str', '*blk': 'bool', '*inc': 'bool', '*detach': 'bool' } }
+ 'data': { 'uri': 'str', '*blk': 'bool', '*inc': 'bool', '*dirty': 'boo
[Qemu-devel] [PATCH RFC v2 2/8] hbitmap: serialization
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bitmap migration, restoring
the bitmap in several steps is available. To save performance, every
step writes only the last level of the bitmap. All other levels are
restored by hbitmap_deserialize_finish() as a last step of restoring.
So, HBitmap is inconsistent while restoring.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/qemu/hbitmap.h | 59 +++
util/hbitmap.c | 96 ++
2 files changed, 155 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index c48c50a..1d37582 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -137,6 +137,65 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t
count);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_data_size:
+ * @hb: HBitmap to operate on.
+ * @count: Number of bits
+ *
+ * Return amount of bytes hbitmap_serialize_part needs
+ */
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count);
+
+/**
+ * hbitmap_serialize_part
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to store serialized bitmap.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
+ * independently of endianess and size of HBitmap level array elements
+ */
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Retores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_serialize_part.
+ *
+ * ! The bitmap becomes inconsistent after this operation.
+ * hbitmap_serialize_finish should be called before using the bitmap after
+ * data restoring.
+ */
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part0
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Same as hbitmap_serialize_part, but fills the bitmap with zeroes.
+ */
+void hbitmap_deserialize_part0(HBitmap *hb, uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
+ * layers are restored here.
+ */
+void hbitmap_deserialize_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index f400dcb..55226a0 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -366,6 +366,102 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size, gran;
+
+if (count == 0) {
+return 0;
+}
+
+gran = 1ll << hb->granularity;
+size = (((gran + count - 2) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *out = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+out[i] = (BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
+}
+}
+
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+uint64_t i;
+uint64_t last = start + count - 1;
+unsigned long *in = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+for (i = start; i <= last; ++i) {
+hb->levels[HBITMAP_LEVELS - 1][i] =
+(BITS_PER_LONG == 32 ? le32_to_cpu(in[i]) : le64_to_cpu(in[i]));
+}
+}
+
+void hbitmap_deserialize_part0(HBitmap *hb, uint64_t start, uint64_t co
[Qemu-devel] [PATCH RFC v2 5/8] block: add bdrv_dirty_bitmap_enabled()
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 5 +
include/block/block.h | 1 +
2 files changed, 6 insertions(+)
diff --git a/block.c b/block.c
index 8ab724b..15fc621 100644
--- a/block.c
+++ b/block.c
@@ -5551,6 +5551,11 @@ uint64_t bdrv_dirty_bitmap_granularity(BlockDriverState
*bs,
return bitmap->granularity;
}
+bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap)
+{
+return bitmap->enabled;
+}
+
void bdrv_dirty_iter_init(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
{
diff --git a/include/block/block.h b/include/block/block.h
index 648b0a9..7b49d98 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -449,6 +449,7 @@ BlockDirtyInfoList
*bdrv_query_dirty_bitmaps(BlockDriverState *bs);
uint64_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
uint64_t bdrv_dirty_bitmap_granularity(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap);
+bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap);
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, int64_t
sector);
void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors);
--
1.9.1
[Qemu-devel] [PATCH RFC v2 1/8] qmp: print dirty bitmap
Adds qmp and hmp commands to print dirty bitmap. This is needed only for
testing.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 33 +
blockdev.c| 13 +
hmp-commands.hx | 15 +++
hmp.c | 8
hmp.h | 1 +
include/block/block.h | 2 ++
qapi-schema.json | 3 ++-
qapi/block-core.json | 3 +++
qmp-commands.hx | 5 +
9 files changed, 82 insertions(+), 1 deletion(-)
diff --git a/block.c b/block.c
index 2466ba8..6d3f0b2 100644
--- a/block.c
+++ b/block.c
@@ -5374,6 +5374,39 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
}
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+unsigned long a = 0, b = 0;
+
+printf("bitmap '%s'\n", bitmap->name ? bitmap->name : "no name");
+printf("enabled: %s\n", bitmap->enabled ? "true" : "false");
+printf("size: %" PRId64 "\n", bitmap->size);
+printf("granularity: %" PRId64 "\n", bitmap->granularity);
+printf("dirty regions begin:\n");
+
+while (true) {
+for (a = b; a < bitmap->size && !hbitmap_get(bitmap->bitmap, a); ++a) {
+;
+}
+if (a >= bitmap->size) {
+break;
+}
+
+for (b = a + 1;
+ b < bitmap->size && hbitmap_get(bitmap->bitmap, b);
+ ++b) {
+;
+}
+
+printf("%ld -> %ld\n", a, b - 1);
+if (b >= bitmap->size) {
+break;
+}
+}
+
+printf("dirty regions end\n");
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
int granularity,
const char *name,
diff --git a/blockdev.c b/blockdev.c
index 209fedd..66f0437 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2074,6 +2074,19 @@ void qmp_block_dirty_bitmap_add(const char *node_ref,
const char *name,
aio_context_release(aio_context);
}
+void qmp_block_dirty_bitmap_print(const char *node_ref, const char *name,
+ Error **errp)
+{
+BdrvDirtyBitmap *bitmap;
+
+bitmap = block_dirty_bitmap_lookup(node_ref, name, NULL, errp);
+if (!bitmap) {
+return;
+}
+
+bdrv_print_dirty_bitmap(bitmap);
+}
+
void qmp_block_dirty_bitmap_remove(const char *node_ref, const char *name,
Error **errp)
{
diff --git a/hmp-commands.hx b/hmp-commands.hx
index e37bc8b..a9be506 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -58,6 +58,21 @@ Quit the emulator.
ETEXI
{
+.name = "print_dirty_bitmap",
+.args_type = "device:B,bitmap:s",
+.params = "device bitmap",
+.help = "print dirty bitmap",
+.user_print = monitor_user_noop,
+.mhandler.cmd = hmp_print_dirty_bitmap,
+},
+
+STEXI
+@item print_dirty_bitmap device_id bitmap_name
+@findex print_dirty_bitmap
+Print dirty bitmap meta information and dirty regions.
+ETEXI
+
+{
.name = "block_resize",
.args_type = "device:B,size:o",
.params = "device size",
diff --git a/hmp.c b/hmp.c
index 63b19c7..a269145 100644
--- a/hmp.c
+++ b/hmp.c
@@ -782,6 +782,14 @@ void hmp_info_tpm(Monitor *mon, const QDict *qdict)
qapi_free_TPMInfoList(info_list);
}
+void hmp_print_dirty_bitmap(Monitor *mon, const QDict *qdict)
+{
+const char *device = qdict_get_str(qdict, "device");
+const char *name = qdict_get_str(qdict, "bitmap");
+
+qmp_block_dirty_bitmap_print(device, name, NULL);
+}
+
void hmp_quit(Monitor *mon, const QDict *qdict)
{
monitor_suspend(mon);
diff --git a/hmp.h b/hmp.h
index 4bb5dca..6bbbc33 100644
--- a/hmp.h
+++ b/hmp.h
@@ -19,6 +19,7 @@
#include "qapi-types.h"
#include "qapi/qmp/qdict.h"
+void hmp_print_dirty_bitmap(Monitor *mon, const QDict *qdict);
void hmp_info_name(Monitor *mon, const QDict *qdict);
void hmp_info_version(Monitor *mon, const QDict *qdict);
void hmp_info_kvm(Monitor *mon, const QDict *qdict);
diff --git a/include/block/block.h b/include/block/block.h
index cb1f28d..6cf067f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -459,6 +459,8 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
diff --git a/qapi-schema.json b/qapi-schema.json
index 85f55d9.
[Qemu-devel] [PATCH RFC v2 8/8] migration: add migration/dirty-bitmap.c
Live migration of dirty bitmaps. Only named dirty bitmaps are migrated.
If destination qemu is already containing a dirty bitmap with the same
name as a migrated bitmap, then their granularities should be the same,
otherwise the error will be generated. If destination qemu doesn't
contain such bitmap it will be created.
format:
1 byte: flags
[ 1 byte: node name size ] \ flags & DEVICE_NAME
[ n bytes: node name ] /
[ 1 byte: bitmap name size ] \
[ n bytes: bitmap name ] | flags & BITMAP_NAME
[ [ be64: granularity] ] flags & GRANULARITY
[ 1 byte: bitmap enabled bit ] flags & ENABLED
[ be64: start sector ] \ flags & (NORMAL_CHUNK | ZERO_CHUNK)
[ be32: number of sectors ] /
[ be64: buffer size ] \ flags & NORMAL_CHUNK
[ n bytes: buffer ] /
The last chunk should contain flags & EOS. The chunk may skip device
and/or bitmap names, assuming them to be the same with the previous
chunk. GRANULARITY is sent with the first chunk for the bitmap. ENABLED
bit is sent in the end of "complete" stage of migration. So when
destination gets ENABLED flag it should deserialize_finish the bitmap
and set its enabled bit to corresponding value.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/migration/block.h | 1 +
migration/Makefile.objs | 2 +-
migration/dirty-bitmap.c | 606 ++
vl.c | 1 +
4 files changed, 609 insertions(+), 1 deletion(-)
create mode 100644 migration/dirty-bitmap.c
diff --git a/include/migration/block.h b/include/migration/block.h
index ffa8ac0..566bb9f 100644
--- a/include/migration/block.h
+++ b/include/migration/block.h
@@ -14,6 +14,7 @@
#ifndef BLOCK_MIGRATION_H
#define BLOCK_MIGRATION_H
+void dirty_bitmap_mig_init(void);
void blk_mig_init(void);
int blk_mig_active(void);
uint64_t blk_mig_bytes_transferred(void);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index d929e96..9adfda9 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -6,5 +6,5 @@ common-obj-y += xbzrle.o
common-obj-$(CONFIG_RDMA) += rdma.o
common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
-common-obj-y += block.o
+common-obj-y += block.o dirty-bitmap.o
diff --git a/migration/dirty-bitmap.c b/migration/dirty-bitmap.c
new file mode 100644
index 000..8621218
--- /dev/null
+++ b/migration/dirty-bitmap.c
@@ -0,0 +1,606 @@
+/*
+ * QEMU dirty bitmap migration
+ *
+ * derived from migration/block.c
+ *
+ * Author:
+ * Sementsov-Ogievskiy Vladimir
+ *
+ * original copyright message:
+ * =
+ * Copyright IBM, Corp. 2009
+ *
+ * Authors:
+ * Liran Schour
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2. See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ * =
+ */
+
+#include "block/block.h"
+#include "qemu/main-loop.h"
+#include "qemu/error-report.h"
+#include "migration/block.h"
+#include "migration/migration.h"
+#include "qemu/hbitmap.h"
+#include
+
+#define CHUNK_SIZE (1 << 20)
+
+#define DIRTY_BITMAP_MIG_FLAG_EOS 0x01
+#define DIRTY_BITMAP_MIG_FLAG_NORMAL_CHUNK 0x02
+#define DIRTY_BITMAP_MIG_FLAG_ZERO_CHUNK0x04
+#define DIRTY_BITMAP_MIG_FLAG_BITMAP_NAME 0x08
+#define DIRTY_BITMAP_MIG_FLAG_DEVICE_NAME 0x10
+#define DIRTY_BITMAP_MIG_FLAG_GRANULARITY 0x20
+#define DIRTY_BITMAP_MIG_FLAG_ENABLED 0x40
+/* flags should be <= 0xff */
+
+/* #define DEBUG_DIRTY_BITMAP_MIGRATION */
+
+#ifdef DEBUG_DIRTY_BITMAP_MIGRATION
+#define DPRINTF(fmt, ...) \
+do { printf("dirty_migration: " fmt, ## __VA_ARGS__); } while (0)
+#else
+#define DPRINTF(fmt, ...) \
+do { } while (0)
+#endif
+
+typedef struct DirtyBitmapMigBitmapState {
+/* Written during setup phase. */
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+HBitmap *dirty_bitmap;
+int64_t total_sectors;
+uint64_t sectors_per_chunk;
+QSIMPLEQ_ENTRY(DirtyBitmapMigBitmapState) entry;
+
+/* For bulk phase. */
+bool bulk_completed;
+int64_t cur_sector;
+bool granularity_sent;
+
+/* For dirty phase. */
+int64_t cur_dirty;
+} DirtyBitmapMigBitmapState;
+
+typedef struct DirtyBitmapMigState {
+int migration_enable;
+QSIMPLEQ_HEAD(dbms_list, DirtyBitmapMigBitmapState) dbms_list;
+
+bool bulk_completed;
+
+/* for send_bitmap() */
+BlockDriverState *prev_bs;
+BdrvDirtyBitmap *prev_bitmap;
+} DirtyBitmapMigState;
+
+static DirtyBitmapMigState dirty_bitmap_mig_state;
+
+/* read name from qemu file:
+ * format:
+ * 1 byte : len = name length (<256)
+ * len bytes : name without last zero byte
+ *
[Qemu-devel] [PATCH RFC v2 0/8] Dirty bitmaps migration
v2: 1. bug-fixes, that are already in upstream, and renaming of function bdrv_reset_dirty_bitmap (which is already in Snow's series) are dropped 2. bitmap store/restore: the concept renamed to serialization, added function hbitmap_deserialize_part0, to not transfer zero blocks 3. migration dirty parameter: added description comment 4. Other patches are new. v2.rfc: Actually, in this version of the series I'm trying not use migration/block.c at all. Instead a separate migration unit is added in the new file migration/dirty-bitmap.c. Now bitmaps are migrated like blocks in block migration, they have their "dirty-dirty" bitmaps, for tracking set/unset changes during migration. The advantages are: - no complications of migration/block.c - separate dirty-dirty bitmaps provide handling of "unset's" - more effective meta-data/data ratio - no tiny bitmap-blocks. v1: These patches provide dirty bitmap migration feature. Only named dirty bitmaps are to be migrated. Migration is made as a part of block migration in block-migration.c. Dirty bitmap migration may be enabled by "dirty" parameter for qmp migrate command. If "blk" and "inc" parameters are false when "dirty" is true block migration is actually skipped: no allocatoions, no bdrv_read's, no bdrv_write's, only bitmaps are migrated. The patch set includes two my previous bug fixes, which are necessary for it. The patch set is based on Incremental backup series by John Snow. Vladimir Sementsov-Ogievskiy (8): qmp: print dirty bitmap hbitmap: serialization block: BdrvDirtyBitmap serialization interface block: add dirty-dirty bitmaps block: add bdrv_dirty_bitmap_enabled() block: add bdrv_next_dirty_bitmap() migration: add dirty parameter migration: add migration/dirty-bitmap.c block.c | 128 + blockdev.c| 13 + hmp-commands.hx | 27 +- hmp.c | 12 +- hmp.h | 1 + include/block/block.h | 22 ++ include/migration/block.h | 1 + include/migration/migration.h | 1 + include/qemu/hbitmap.h| 59 migration/Makefile.objs | 2 +- migration/dirty-bitmap.c | 606 ++ migration/migration.c | 4 +- qapi-schema.json | 10 +- qapi/block-core.json | 3 + qmp-commands.hx | 10 +- savevm.c | 3 +- util/hbitmap.c| 96 +++ vl.c | 1 + 18 files changed, 987 insertions(+), 12 deletions(-) create mode 100644 migration/dirty-bitmap.c -- 1.9.1
Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)
ping Best regards, Vladimir On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other formats we can store bitmaps in the separate qcow2 file QCow2 header is extended by fields 'nb_dirty_bitmaps' and 'dirty_bitmaps_offset' like with snapshots. Proposed command line syntax is the following: -dirty-bitmap [option1=val1][,option2=val2]... Available options are: name The name for the bitmap (necessary). file The file to load the bitmap from. file_id When specified with 'file' option, then this file will be available through this id for other -dirty-bitmap options when specified without 'file' option, then it is a reference to 'file', specified with another -dirty-bitmap option, and it will be used to load the bitmap from. driveThe drive to bind the bitmap to. It should be specified as 'id' suboption of one of -drive options. If nor 'file' neither 'file_id' are specified, then the bitmap will be loaded from that drive (internal dirty bitmap). granularity The granularity for the bitmap. Not necessary, the default value may be used. enabled on|off. Default is 'on'. Disabled bitmaps are not changing regardless of writes to corresponding drive. Examples: qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk qemu -drive file=a.raw,id=disk \ -dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off Vladimir Sementsov-Ogievskiy (8): spec: add qcow2-dirty-bitmaps specification hbitmap: store / restore qcow2: add dirty-bitmaps feature block: store persistent dirty bitmaps block: add bdrv_load_dirty_bitmap qemu: command line option for dirty bitmaps qmp: print dirty bitmap iotests: test internal persistent dirty bitmap block.c| 113 ++ block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 514 + block/qcow2.c | 26 +++ block/qcow2.h | 48 + blockdev.c | 51 + docs/specs/qcow2.txt | 59 ++ hmp-commands.hx| 15 ++ hmp.c | 8 + hmp.h | 1 + include/block/block.h | 9 + include/block/block_int.h | 10 + include/qemu/hbitmap.h | 49 + include/sysemu/blockdev.h | 1 + include/sysemu/sysemu.h| 1 + qapi-schema.json | 3 +- qapi/block-core.json | 3 + qemu-options.hx| 37 qmp-commands.hx| 5 + tests/qemu-iotests/115 | 96 + tests/qemu-iotests/115.out | 64 ++ tests/qemu-iotests/group | 1 + util/hbitmap.c | 87 vl.c | 100 + 24 files changed, 1301 insertions(+), 2 deletions(-) create mode 100644 block/qcow2-dirty-bitmap.c create mode 100755 tests/qemu-iotests/115 create mode 100644 tests/qemu-iotests/115.out
Re: [Qemu-devel] [PATCH RFC v2 1/8] qmp: print dirty bitmap
Ok, I agree. It was just a simple way to test the other staff. I'll
rewrite it in the following versions of my two series.
Best regards,
Vladimir
On 27.01.2015 19:17, Eric Blake wrote:
On 01/27/2015 03:56 AM, Vladimir Sementsov-Ogievskiy wrote:
Adds qmp and hmp commands to print dirty bitmap. This is needed only for
testing.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 33 +
blockdev.c| 13 +
hmp-commands.hx | 15 +++
hmp.c | 8
hmp.h | 1 +
include/block/block.h | 2 ++
qapi-schema.json | 3 ++-
qapi/block-core.json | 3 +++
qmp-commands.hx | 5 +
9 files changed, 82 insertions(+), 1 deletion(-)
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+unsigned long a = 0, b = 0;
+
+printf("bitmap '%s'\n", bitmap->name ? bitmap->name : "no name");
+printf("enabled: %s\n", bitmap->enabled ? "true" : "false");
+printf("size: %" PRId64 "\n", bitmap->size);
+printf("granularity: %" PRId64 "\n", bitmap->granularity);
+printf("dirty regions begin:\n");
+
+void qmp_block_dirty_bitmap_print(const char *node_ref, const char *name,
+ Error **errp)
+{
+BdrvDirtyBitmap *bitmap;
+
+bitmap = block_dirty_bitmap_lookup(node_ref, name, NULL, errp);
+if (!bitmap) {
+return;
+}
+
+bdrv_print_dirty_bitmap(bitmap);
Won't work. You cannot assume that stdout is usable when invoked from
QMP. The only sane thing to do is to bundle up the structured data into
JSON, pass that back over the QMP connection, and let the client decide
how to print it.
I'm opposed to adding this command as-is.
Re: [Qemu-devel] [PATCH] block-migration: fix pending() return value
v2 is already pushed http://git.qemu.org/?p=qemu.git;a=commit;h=04636dc410b163c2243e66c3813dd4900a50a4ed Best regards, Vladimir On 29.01.2015 10:58, Markus Armbruster wrote: Vladimir Sementsov-Ogievskiy writes: Hi there.. Can someone review my bug fix? I'll surely fix typos in description after it. The file you patch has since moved. Suggest a quick rebase. Make sure to cc: your v2 to migration maintainers. $ scripts/get_maintainer.pl -f migration/block.c Kevin Wolf (supporter:Block) Stefan Hajnoczi (supporter:Block) Juan Quintela (maintainer:Migration) Amit Shah (maintainer:Migration)
Re: [Qemu-devel] [PATCH v11 03/13] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
s/{'command'/{ 'command'/g
- to be consistent with other staff in qapi/block-core.json
and the same fix for block-dirty-bitmap-enable and
block-dirty-bitmap-disable
Best regards,
Vladimir
On 12.01.2015 19:30, John Snow wrote:
From: Fam Zheng
The new command pair is added to manage user created dirty bitmap. The
dirty bitmap's name is mandatory and must be unique for the same device,
but different devices can have bitmaps with the same names.
The granularity is an optional field. If it is not specified, we will
choose a default granularity based on the cluster size if available,
clamped to between 4K and 64K to mirror how the 'mirror' code was
already choosing granularity. If we do not have cluster size info
available, we choose 64K. This code has been factored out into a helper
shared with block/mirror.
This patch also introduces the 'block_dirty_bitmap_lookup' helper,
which takes a device name and a dirty bitmap name and validates the
lookup, returning NULL and setting errp if there is a problem with
either field. This helper will be re-used in future patches in this
series.
The types added to block-core.json will be re-used in future patches
in this series, see:
'qapi: Add transaction support to block-dirty-bitmap-{add, enable, disable}'
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
---
block.c | 20 ++
block/mirror.c| 10 +
blockdev.c| 100 ++
include/block/block.h | 1 +
qapi/block-core.json | 55 +++
qmp-commands.hx | 51 +
6 files changed, 228 insertions(+), 9 deletions(-)
diff --git a/block.c b/block.c
index bfeae6b..3eb77ee 100644
--- a/block.c
+++ b/block.c
@@ -5417,6 +5417,26 @@ int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap, int64_t sector
}
}
+/**
+ * Chooses a default granularity based on the existing cluster size,
+ * but clamped between [4K, 64K]. Defaults to 64K in the case that there
+ * is no cluster size information available.
+ */
+uint64_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
+{
+BlockDriverInfo bdi;
+uint64_t granularity;
+
+if (bdrv_get_info(bs, &bdi) >= 0 && bdi.cluster_size != 0) {
+granularity = MAX(4096, bdi.cluster_size);
+granularity = MIN(65536, granularity);
+} else {
+granularity = 65536;
+}
+
+return granularity;
+}
+
void bdrv_dirty_iter_init(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
{
diff --git a/block/mirror.c b/block/mirror.c
index d819952..fc545f1 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -667,15 +667,7 @@ static void mirror_start_job(BlockDriverState *bs,
BlockDriverState *target,
MirrorBlockJob *s;
if (granularity == 0) {
-/* Choose the default granularity based on the target file's cluster
- * size, clamped between 4k and 64k. */
-BlockDriverInfo bdi;
-if (bdrv_get_info(target, &bdi) >= 0 && bdi.cluster_size != 0) {
-granularity = MAX(4096, bdi.cluster_size);
-granularity = MIN(65536, granularity);
-} else {
-granularity = 65536;
-}
+granularity = bdrv_get_default_bitmap_granularity(target);
}
assert ((granularity & (granularity - 1)) == 0);
diff --git a/blockdev.c b/blockdev.c
index 5651a8e..95251c7 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1173,6 +1173,48 @@ out_aio_context:
return NULL;
}
+/**
+ * Return a dirty bitmap (if present), after validating
+ * the node reference and bitmap names. Returns NULL on error,
+ * including when the BDS and/or bitmap is not found.
+ */
+static BdrvDirtyBitmap *block_dirty_bitmap_lookup(const char *node_ref,
+ const char *name,
+ BlockDriverState **pbs,
+ Error **errp)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+
+if (!node_ref) {
+error_setg(errp, "Node reference cannot be NULL");
+return NULL;
+}
+if (!name) {
+error_setg(errp, "Bitmap name cannot be NULL");
+return NULL;
+}
+
+bs = bdrv_lookup_bs(node_ref, node_ref, errp);
+if (!bs) {
+error_setg(errp, "Node reference '%s' not found", node_ref);
+return NULL;
+}
+
+/* If caller provided a BDS*, provide the result of that lookup, too. */
+if (pbs) {
+*pbs = bs;
+}
+
+bitmap = bdrv_find_dirty_bitmap(bs, name);
+if (!bitmap) {
+error_setg(errp, "Dirty bitmap not found: %s", name);
+return NULL;
+}
+
+return bitmap;
+}
+
/* New and old BlockDriverState structs for atomic group operations */
typedef struct BlkTransactionState BlkTransactionState;
@@ -1894,6 +1936,64 @@ void qmp_block_set_io_throttle(const
Re: [Qemu-devel] [PATCH 7/8] qmp: print dirty bitmap
is it better to add qmp_query_dirty_bitmap with underlying
bdrv_query_dirty_bitmap, or to modify (add dirty regions information)
existing qmp_query_block/qmp_query_dirty_bitmapS?
Best regards,
Vladimir
On 27.01.2015 18:53, Eric Blake wrote:
On 01/13/2015 10:02 AM, Vladimir Sementsov-Ogievskiy wrote:
Adds qmp and hmp commands to print dirty bitmap. This is needed only for
testing persistent dirty bitmap feature.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
+++ b/block.c
@@ -5445,6 +5445,39 @@ int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap)
return res;
}
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+unsigned long a = 0, b = 0;
+
+printf("bitmap '%s'\n", bitmap->name ? bitmap->name : "no name");
+printf("enabled: %s\n", bitmap->enabled ? "true" : "false");
+printf("size: %" PRId64 "\n", bitmap->size);
+printf("granularity: %" PRId64 "\n", bitmap->granularity);
+printf("dirty regions begin:\n");
+++ b/blockdev.c
@@ -2079,6 +2079,19 @@ void qmp_block_dirty_bitmap_add(const char *node_ref,
const char *name,
aio_context_release(aio_context);
}
+void qmp_block_dirty_bitmap_print(const char *node_ref, const char *name,
+ Error **errp)
+{
+BdrvDirtyBitmap *bitmap;
+
+bitmap = block_dirty_bitmap_lookup(node_ref, name, NULL, errp);
+if (!bitmap) {
+return;
+}
+
+bdrv_print_dirty_bitmap(bitmap);
Eww. This assumes that stdout is usable. But that is not the case for
QMP commands. It would be better to package up the output in a
structured format and return it to the caller, and let the caller decide
how to print it.
+++ b/qapi/block-core.json
@@ -982,6 +982,9 @@
{'command': 'block-dirty-bitmap-disable',
'data': 'BlockDirtyBitmap' }
+{'command': 'block-dirty-bitmap-print',
+ 'data': 'BlockDirtyBitmap' }
Missing documentation, if we even want this command. As mentioned
above, this should return ALL of the information necessary for the
client to then decide how to print the information, rather than directly
printing information to stdout itself. And it might be better to name
this command in the 'query-' namespace.
Re: [Qemu-devel] [PATCH v11 00/13] block: Incremental backup series
About added functions for BdrvDirtyBitmap:
some functions has needless BlockDriverState* parameter, and others -
doesn't:
with needless *bs:
bdrv_dirty_bitmap_make_anon
bdrv_dirty_bitmap_granularity
bdrv_clear_dirty_bitmap
without *bs:
bdrv_dirty_bitmap_enabled
bdrv_disable_dirty_bitmap
bdrv_enable_dirty_bitmap
I think, to be consistent, onlly one of two ways should be chosen:
1) all bdrv_dirty_bitmap_* functions has "BlockDriverState* bs"
parameter, maybe needless
2) only bdrv_dirty_bitmap_* function that need "BlockDriverState* bs"
parameter has it
Personally, I prefer the second one.
Best regards,
Vladimir
On 12.01.2015 19:30, John Snow wrote:
Welcome to version 11. I hope you are enjoying our regular newsletter.
This patchset enables the in-memory part of the incremental backup
feature. A patchset by Vladimir Sementsov-Ogievskiy enables the
migration of in-memory dirty bitmaps, and a future patchset will
enable the storage and retrieval of dirty bitmaps to and from permanent
storage.
Enough changes have been made that most Reviewed-By lines from
previous iterations have been removed. (Sorry!)
This series was originally authored by Fam Zheng;
his cover letter is included below.
~John Snow
=
This is the in memory part of the incremental backup feature.
With the added commands, we can create a bitmap on a block
backend, from which point of time all the writes are tracked by
the bitmap, marking sectors as dirty. Later, we call drive-backup
and pass the bitmap to it, to do an incremental backup.
See the last patch which adds some tests for this use case.
Fam
=
For convenience, this patchset is available on github:
https://github.com/jnsnow/qemu/commits/dbm-backup
v11:
- Instead of copying BdrvDirtyBitmaps and keeping a pointer to the
object we were copied from, we instead "freeze" a bitmap in-place
without copying it. On success, we thaw and delete the bitmap.
On failure, we merge the bitmap with a "successor," which is an
anonymous bitmap attached as a child that records writes for us
for the duration of the backup operation.
This means that incremental backups can NEVER BE RETRIED in a
deterministic fashion. If an incremental backup fails on a set
of dirty sectors {x}, and a new set of dirty sectors {y} are
introduced during the backup, then any possible retry action
on an incremental backup can only operate on {x,y}. There is no
way to get an incremental backup "as it would have been."
So, the failure mode for incremental backup is to try again,
and the resulting image will simply be a differential from the
last successful dirty bitmap backup.
- Removed hbitmap_copy and bdrv_dirty_bitmap_copy.
- Added a small fixup patch:
- Update all granularity fields to be uint64_t.
- Update documentation around BdrvDirtyBitmap structure.
- Modified transactions to obey frozen attribute of BdrvDirtyBitmaps.
- Added frozen attribute to the info query.
v10 (Overview):
- I've included one of Vladimir's bitmap fixes as patch #1.
- QMP commands and transactions are now protected via
aio_context functions.
- QMP commands use "node-ref" as a parameter name now. Reasoning
is thus: Either a "device name" or a "node name" can be used to
reference a BDS, which is the key item we are actually acting
on with these bitmap commands. Thus, I refer to this unified
parameter as a "Node Reference," or "node-ref."
We could also argue for "backend-ref" or "device-ref" for the
reverse semantics: where we accept a unified parameter, but we
intend to resolve it to the BlockBackend instead of resolving
the parameter given to the BlockDriverState.
Or, we could use "reference" for both cases, and to match the
existing BlockdevRef command.
- All QMP commands added are now per-node via a unified parameter
name. Though backup only operates on "devices," you are free to
create bitmaps for any arbitrary node you wish. It is obviously
only useful for the root node, currently.
- Bitmap Sync modes (CONSUME, RESET) are removed. See below
(changelog, RFC questions) for more details.
- Code to recover the bitmap after a failure has been added,
but I have some major questions about drive_backup guarantees.
See RFC questions.
v10 (Detailed Changelog):
(1/13) New Patch:
- Included Vladimir Sementsov-Ogievskiy's patch that clarifies
the semantics of the bitmap primitives.
(3/13):
- Edited function names for consistency (Stefanha)
- Removed compile-time constants (Stefanha)
- Acquire aio_context for bitmap add/remove (Stefanha)
- Documented
Re: [Qemu-devel] [PATCH RFC v2 7/8] migration: add dirty parameter
On 27.01.2015 19:20, Eric Blake wrote: On 01/27/2015 03:56 AM, Vladimir Sementsov-Ogievskiy wrote: Add dirty parameter to qmp-migrate command. If this parameter is true, migration/block.c will migrate dirty bitmaps. This parameter can be used without "blk" parameter to migrate only dirty bitmaps, skipping block migration. Signed-off-by: Vladimir Sementsov-Ogievskiy --- +++ b/qapi-schema.json @@ -1656,12 +1656,17 @@ # @detach: this argument exists only for compatibility reasons and # is ignored by QEMU # +# @dirty: #optional do dirty-bitmaps migration (can be used with or without +# @blk parameter) +# (since 2.3) Rather than adding it to 'migrate', where the command is not introspectible, I'd rather you add it to 'migrate-set-capabilities', where I can then use 'query-migrate-capabilities' to learn if it is supported. Thank you. Moreover, it is turned out to be a simpler way than with migration parameter. I've done the change, it will be in v3. I'm just waiting for review of the core part of the series - migration/dirty-bitmap.c -- Best regards, Vladimir
Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)
On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other formats we can store bitmaps in the separate qcow2 file QCow2 header is extended by fields 'nb_dirty_bitmaps' and 'dirty_bitmaps_offset' like with snapshots. Proposed command line syntax is the following: -dirty-bitmap [option1=val1][,option2=val2]... Available options are: name The name for the bitmap (necessary). file The file to load the bitmap from. file_id When specified with 'file' option, then this file will be available through this id for other -dirty-bitmap options when specified without 'file' option, then it is a reference to 'file', specified with another -dirty-bitmap option, and it will be used to load the bitmap from. driveThe drive to bind the bitmap to. It should be specified as 'id' suboption of one of -drive options. If nor 'file' neither 'file_id' are specified, then the bitmap will be loaded from that drive (internal dirty bitmap). granularity The granularity for the bitmap. Not necessary, the default value may be used. enabled on|off. Default is 'on'. Disabled bitmaps are not changing regardless of writes to corresponding drive. Examples: qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk qemu -drive file=a.raw,id=disk \ -dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off Vladimir Sementsov-Ogievskiy (8): spec: add qcow2-dirty-bitmaps specification hbitmap: store / restore qcow2: add dirty-bitmaps feature block: store persistent dirty bitmaps block: add bdrv_load_dirty_bitmap qemu: command line option for dirty bitmaps qmp: print dirty bitmap iotests: test internal persistent dirty bitmap block.c| 113 ++ block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 514 + block/qcow2.c | 26 +++ block/qcow2.h | 48 + blockdev.c | 51 + docs/specs/qcow2.txt | 59 ++ hmp-commands.hx| 15 ++ hmp.c | 8 + hmp.h | 1 + include/block/block.h | 9 + include/block/block_int.h | 10 + include/qemu/hbitmap.h | 49 + include/sysemu/blockdev.h | 1 + include/sysemu/sysemu.h| 1 + qapi-schema.json | 3 +- qapi/block-core.json | 3 + qemu-options.hx| 37 qmp-commands.hx| 5 + tests/qemu-iotests/115 | 96 + tests/qemu-iotests/115.out | 64 ++ tests/qemu-iotests/group | 1 + util/hbitmap.c | 87 vl.c | 100 + 24 files changed, 1301 insertions(+), 2 deletions(-) create mode 100644 block/qcow2-dirty-bitmap.c create mode 100755 tests/qemu-iotests/115 create mode 100644 tests/qemu-iotests/115.out Ping. I've already done (locally): 1) using qcow2 header extension instead of changing the header 2) normal qmp query request instead of "print dirty bitmap" - thanks to Eric and Markus Now I'm waiting for some comments on the concept and it's realization to roll v3. -- Best regards, Vladimir
Re: [Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)
On 04.02.2015 18:20, John Snow wrote: On 02/04/2015 10:07 AM, Vladimir Sementsov-Ogievskiy wrote: On 13.01.2015 20:02, Vladimir Sementsov-Ogievskiy wrote: The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other formats we can store bitmaps in the separate qcow2 file QCow2 header is extended by fields 'nb_dirty_bitmaps' and 'dirty_bitmaps_offset' like with snapshots. Proposed command line syntax is the following: -dirty-bitmap [option1=val1][,option2=val2]... Available options are: name The name for the bitmap (necessary). file The file to load the bitmap from. file_id When specified with 'file' option, then this file will be available through this id for other -dirty-bitmap options when specified without 'file' option, then it is a reference to 'file', specified with another -dirty-bitmap option, and it will be used to load the bitmap from. driveThe drive to bind the bitmap to. It should be specified as 'id' suboption of one of -drive options. If nor 'file' neither 'file_id' are specified, then the bitmap will be loaded from that drive (internal dirty bitmap). granularity The granularity for the bitmap. Not necessary, the default value may be used. enabled on|off. Default is 'on'. Disabled bitmaps are not changing regardless of writes to corresponding drive. Examples: qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk qemu -drive file=a.raw,id=disk \ -dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off Vladimir Sementsov-Ogievskiy (8): spec: add qcow2-dirty-bitmaps specification hbitmap: store / restore qcow2: add dirty-bitmaps feature block: store persistent dirty bitmaps block: add bdrv_load_dirty_bitmap qemu: command line option for dirty bitmaps qmp: print dirty bitmap iotests: test internal persistent dirty bitmap block.c| 113 ++ block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 514 + block/qcow2.c | 26 +++ block/qcow2.h | 48 + blockdev.c | 51 + docs/specs/qcow2.txt | 59 ++ hmp-commands.hx| 15 ++ hmp.c | 8 + hmp.h | 1 + include/block/block.h | 9 + include/block/block_int.h | 10 + include/qemu/hbitmap.h | 49 + include/sysemu/blockdev.h | 1 + include/sysemu/sysemu.h| 1 + qapi-schema.json | 3 +- qapi/block-core.json | 3 + qemu-options.hx| 37 qmp-commands.hx| 5 + tests/qemu-iotests/115 | 96 + tests/qemu-iotests/115.out | 64 ++ tests/qemu-iotests/group | 1 + util/hbitmap.c | 87 vl.c | 100 + 24 files changed, 1301 insertions(+), 2 deletions(-) create mode 100644 block/qcow2-dirty-bitmap.c create mode 100755 tests/qemu-iotests/115 create mode 100644 tests/qemu-iotests/115.out Ping. I've already done (locally): 1) using qcow2 header extension instead of changing the header 2) normal qmp query request instead of "print dirty bitmap" - thanks to Eric and Markus Now I'm waiting for some comments on the concept and it's realization to roll v3. My apologies, I've been sick for a while. I'll get going on finishing review of these two series. I assume you'd like to merge the bitmaps migration first? --js The sequence is not important. These series are necessary for making a complete backup solution - don't lose named dirty bitmaps on shutdown and migration. Shutdown is more often task, and dirty bitmaps migration was appeared as a subtask when we were discussed the format for saving dirty bitmaps. But they both are needed, then, ok, let's start with migration. -- Best regards, Vladimir
Re: [Qemu-devel] [PATCH v11 12/13] qemu-iotests: Add tests for drive-backup sync=dirty-bitmap
On 12.01.2015 19:31, John Snow wrote:
From: Fam Zheng
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
---
tests/qemu-iotests/056| 33 ++---
tests/qemu-iotests/056.out| 4 ++--
tests/qemu-iotests/iotests.py | 8
3 files changed, 40 insertions(+), 5 deletions(-)
diff --git a/tests/qemu-iotests/056 b/tests/qemu-iotests/056
index 54e4bd0..6f8aa9a 100755
--- a/tests/qemu-iotests/056
+++ b/tests/qemu-iotests/056
@@ -23,17 +23,17 @@
import time
import os
import iotests
-from iotests import qemu_img, qemu_io, create_image
+from iotests import qemu_img, qemu_img_map_assert, qemu_io, create_image
backing_img = os.path.join(iotests.test_dir, 'backing.img')
test_img = os.path.join(iotests.test_dir, 'test.img')
target_img = os.path.join(iotests.test_dir, 'target.img')
-class TestSyncModesNoneAndTop(iotests.QMPTestCase):
+class TestSyncModes(iotests.QMPTestCase):
image_len = 64 * 1024 * 1024 # MB
def setUp(self):
-create_image(backing_img, TestSyncModesNoneAndTop.image_len)
+create_image(backing_img, TestSyncModes.image_len)
qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' %
backing_img, test_img)
qemu_io('-c', 'write -P0x41 0 512', test_img)
qemu_io('-c', 'write -P0xd5 1M 32k', test_img)
@@ -64,6 +64,33 @@ class TestSyncModesNoneAndTop(iotests.QMPTestCase):
self.assertTrue(iotests.compare_images(test_img, target_img),
'target image does not match source after backup')
+def test_sync_dirty_bitmap_missing(self):
+self.assert_no_active_block_jobs()
+result = self.vm.qmp('drive-backup', device='drive0',
sync='dirty-bitmap',
+ format=iotests.imgfmt, target=target_img)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+def test_sync_dirty_bitmap_not_found(self):
+self.assert_no_active_block_jobs()
+result = self.vm.qmp('drive-backup', device='drive0',
sync='dirty-bitmap',
+ bitmap='unknown',
+ format=iotests.imgfmt, target=target_img)
+self.assert_qmp(result, 'error/class', 'GenericError')
+
+def test_sync_dirty_bitmap(self):
+self.assert_no_active_block_jobs()
+result = self.vm.qmp('block-dirty-bitmap-add', node_ref='drive0',
name='bitmap0')
+self.assert_qmp(result, 'return', {})
+self.vm.hmp_qemu_io('drive0', 'write -P0x5a 0 512')
+self.vm.hmp_qemu_io('drive0', 'write -P0x5a 48M 512')
+result = self.vm.qmp('drive-backup', device='drive0',
sync='dirty-bitmap',
+ bitmap='bitmap0',
+ format=iotests.imgfmt, target=target_img)
+self.assert_qmp(result, 'return', {})
+self.wait_until_completed(check_offset=False)
+self.assert_no_active_block_jobs()
+qemu_img_map_assert(target_img, [0, 0x300])
+
def test_cancel_sync_none(self):
self.assert_no_active_block_jobs()
diff --git a/tests/qemu-iotests/056.out b/tests/qemu-iotests/056.out
index fbc63e6..914e373 100644
--- a/tests/qemu-iotests/056.out
+++ b/tests/qemu-iotests/056.out
@@ -1,5 +1,5 @@
-..
+.
--
-Ran 2 tests
+Ran 5 tests
OK
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index f57f154..95bb959 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -55,6 +55,14 @@ def qemu_img_pipe(*args):
'''Run qemu-img and return its output'''
return subprocess.Popen(qemu_img_args + list(args),
stdout=subprocess.PIPE).communicate()[0]
+def qemu_img_map_assert(img, offsets):
+'''Run qemu-img map on img and check the mapped ranges'''
+offs = []
+for line in qemu_img_pipe('map', img).splitlines()[1:]:
+offset, length, mapped, fname = line.split()
+offs.append(int(offset, 16))
+assert set(offs) == set(offsets), "mapped offsets in image '%s' not equal to
'%s'" % (str(offs), str(offsets))
+
def qemu_io(*args):
'''Run qemu-io and return the stdout data'''
args = qemu_io_args + list(args)
why not just check by qemu-io -c "write -P0x5a 0 512" for symmetry?
-- Best regards, Vladimir
Re: [Qemu-devel] [PATCH v10 00/13] block: Incremental backup series (RFC)
On 23.12.2014 04:12, John Snow wrote: For convenience, this patchset is available on github: https://github.com/jnsnow/qemu/commits/dbm-backup - old version here.
[Qemu-devel] [PATCH v2 1/1] migration/block: fix pending() return value
Because of wrong return value of .save_live_pending() in
migration/block.c, migration finishes before the whole disk is
transferred. Such situation occurs when the migration process is fast
enough, for example when source and dest are on the same host.
If in the bulk phase we return something < max_size, we will skip
transferring the tail of the device. Currently we have "set pending to
BLOCK_SIZE if it is zero" for bulk phase, but there no guarantee, that
it will be < max_size.
True approach is to return, for example, max_size+1 when we are in the
bulk phase.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
migration/block.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/migration/block.c b/migration/block.c
index 74d9eb1..2e92605 100644
--- a/migration/block.c
+++ b/migration/block.c
@@ -765,8 +765,8 @@ static uint64_t block_save_pending(QEMUFile *f, void
*opaque, uint64_t max_size)
block_mig_state.read_done * BLOCK_SIZE;
/* Report at least one block pending during bulk phase */
-if (pending == 0 && !block_mig_state.bulk_completed) {
-pending = BLOCK_SIZE;
+if (pending <= max_size && !block_mig_state.bulk_completed) {
+pending = max_size + BLOCK_SIZE;
}
blk_mig_unlock();
qemu_mutex_unlock_iothread();
--
1.9.1
[Qemu-devel] [PATCH v2 0/1] Fix block migration bug
v2: - rebase to master - fix typos in description Because of wrong return value of .save_live_pending() in block-migration, migration finishes before the whole disk is transferred. Such situation occurs when the migration process is fast enough, for example when source and dest are on the same host. It's easy to test this with the following: bug.sh = #!/bin/sh size=$1 addr=$2 rm /tmp/fifo-mig /tmp/a /tmp/b /tmp/sock-mig ./qemu-img create -f qcow2 /tmp/a $size ./qemu-img create -f qcow2 /tmp/b $size ./qemu-io -c "write -P 0x22 $addr 512" /tmp/a mkfifo /tmp/fifo-mig ./x86_64-softmmu/qemu-system-x86_64 -drive file=/tmp/b,id=disk\ -qmp unix:/tmp/sock-mig,server,nowait\ -incoming "exec: cat /tmp/fifo-mig" & echo 'migrate -b exec:cat>/tmp/fifo-mig\nquit\n' |\ ./x86_64-softmmu/qemu-system-x86_64 -drive file=/tmp/a,id=disk\ -monitor stdio ./scripts/qmp/qmp --path=/tmp/sock-mig quit sleep 3 echo checking ./qemu-io -c "read -P 0x22 $addr 512" /tmp/b = For './bug.sh 1G 1M' qemu-io check finishes successfully, but for './bug.sh 1G 1022M' it finishes with 'Pattern verification failed' status. The following patch fixes this bug. Vladimir Sementsov-Ogievskiy (1): migration/block: fix pending() return value migration/block.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 1.9.1
Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap
I'm sorry if it was already discussed, but I think it is inconsistent to
have "size" in sectors and "granularity" in bytes in one structure. I've
misused these fields because of this in my current work.
At least, I think there should be comments about this.
Best regards,
Vladimir
On 23.12.2014 04:12, John Snow wrote:
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
---
block.c | 39 +++
include/block/block.h | 4
2 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/block.c b/block.c
index a1d9e88..f9e0767 100644
--- a/block.c
+++ b/block.c
@@ -53,6 +53,8 @@
struct BdrvDirtyBitmap {
HBitmap *bitmap;
+int64_t size;
+int64_t granularity;
char *name;
QLIST_ENTRY(BdrvDirtyBitmap) list;
};
@@ -5343,6 +5345,21 @@ void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
bitmap->name = NULL;
}
+BdrvDirtyBitmap *bdrv_copy_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap,
+const char *name)
+{
+BdrvDirtyBitmap *new_bitmap;
+
+new_bitmap = g_malloc0(sizeof(BdrvDirtyBitmap));
+new_bitmap->bitmap = hbitmap_copy(bitmap->bitmap);
+new_bitmap->size = bitmap->size;
+new_bitmap->granularity = bitmap->granularity;
+new_bitmap->name = g_strdup(name);
+QLIST_INSERT_HEAD(&bs->dirty_bitmaps, new_bitmap, list);
+return new_bitmap;
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
int granularity,
const char *name,
@@ -5350,6 +5367,7 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
{
int64_t bitmap_size;
BdrvDirtyBitmap *bitmap;
+int sector_granularity;
assert((granularity & (granularity - 1)) == 0);
@@ -5357,8 +5375,8 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
error_setg(errp, "Bitmap already exists: %s", name);
return NULL;
}
-granularity >>= BDRV_SECTOR_BITS;
-assert(granularity);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
bitmap_size = bdrv_nb_sectors(bs);
if (bitmap_size < 0) {
error_setg_errno(errp, -bitmap_size, "could not get length of
device");
@@ -5366,7 +5384,9 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
return NULL;
}
bitmap = g_new0(BdrvDirtyBitmap, 1);
-bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(granularity) - 1);
+bitmap->size = bitmap_size;
+bitmap->granularity = granularity;
+bitmap->bitmap = hbitmap_alloc(bitmap->size, ffs(sector_granularity) - 1);
bitmap->name = g_strdup(name);
QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
return bitmap;
@@ -5439,7 +5459,9 @@ uint64_t
bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
uint64_t bdrv_dirty_bitmap_granularity(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
{
-return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
+g_assert(BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap) ==
+ bitmap->granularity);
+return bitmap->granularity;
}
void bdrv_dirty_iter_init(BlockDriverState *bs,
@@ -5460,6 +5482,15 @@ void bdrv_reset_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
}
+/**
+ * Effectively, reset the hbitmap from bits [0, size)
+ * Synonymous with bdrv_reset_dirty_bitmap(bs, bitmap, 0, bitmap->size)
+ */
+void bdrv_clear_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
+{
+hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+}
+
static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index c7402e7..e964abd 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -436,6 +436,9 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState
*bs,
BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs,
const char *name);
void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap);
+BdrvDirtyBitmap *bdrv_copy_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap,
+const char *name);
void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
uint64_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
@@ -446,6 +449,7 @@ void bdrv_set_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int nr_sectors);
void
Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration
Best regards, Vladimir On 09.01.2015 01:36, Paolo Bonzini wrote: The bitmaps are transmitted many times in their entirety, but only the last copy actually means something. The others are lost. This means you should use the non-live interface (register_savevm). This will simplify the code a lot. But what if the dirty bitmap is really big? For example, for 5 Pb drive the bitmap with granularity 65536 will be of 2 Gb size. So, shouldn't it be migrated iteratively for live migration?
Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration
On 12.01.2015 17:42, Paolo Bonzini wrote: On 12/01/2015 15:20, Vladimir Sementsov-Ogievskiy wrote: On 09.01.2015 01:36, Paolo Bonzini wrote: The bitmaps are transmitted many times in their entirety, but only the last copy actually means something. The others are lost. This means you should use the non-live interface (register_savevm). This will simplify the code a lot. But what if the dirty bitmap is really big? For example, for 5 Pb drive the bitmap with granularity 65536 will be of 2 Gb size. So, shouldn't it be migrated iteratively for live migration? But your code is not attempting to do that. It is not attempting to track the dirtiness of the dirty bitmap, so to speak. For such a huge storage, in any case, I suspect the solution is to not use QEMU's live operations and, instead, operate at the storage level. Paolo On 12.01.2015 22:09, Paolo Bonzini wrote: On 12/01/2015 18:31, John Snow wrote: How do you track which parts of the disk have been acted upon (e.g. mirrored in the case of the drive-mirror command), so that they have become 0? Hm... so your question here is, "What happens if the named bitmap you are transferring gets reset to 0 during migration? Will anybody track this change?" As far as I know, the only "consumer" of _named_ BdrvDirtyBitmaps is block/backup.c, and we do not support it for mirror (yet?) Right. So can a block-backup job be running WHILE we migrate? I think so. Paolo There is no dirty bitmap for dirty bitmap in my solution, actually block migration dirty bitmap is used for this. It provides tracking of setting bitmap bits to 1 (when corresponding blocks are written), and if the bitmap changes only is this way (no other block-jobs are working with the migrated bitmap) - then the bitmap will be migrated iteratively as it is. Such approach doesn't provide tracking of resetting the bitmap to 0, and such changes are not migrated. So if the bitmap is in use by any block-job, it may have additional set bits after migration. If we want to migrate more precisely we need separate dirty bitmap for every migrated bitmap. And tracking of this dirty-dirty bitmap should be injected into BdrvDirtyBitmap interface. But do we really need precise migration of the bitmap being in use by some block-job (backup for example)? In this case we should "migrate" the backup process to, migrate working block-job.. Is it possible and is it planned to be implemented? Best regards, Vladimir
Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap
Hmm, can't find it in v11
Best regards,
Vladimir
On 10.01.2015 06:25, John Snow wrote:
On 12/30/2014 08:47 AM, Vladimir Sementsov-Ogievskiy wrote:
I'm sorry if it was already discussed, but I think it is inconsistent to
have "size" in sectors and "granularity" in bytes in one structure. I've
misused these fields because of this in my current work.
At least, I think there should be comments about this.
Best regards,
Vladimir
Looking at it now, I think it'd be worse to change it, because it
represents the "size" of the bitmap, as in the number of logical bits
for the interface of the bitmap. Since it is a sector bitmap, I think
this should remain in sectors, for now.
I really want to keep the granularity in bytes in this case, because I
want to match the existing interface, and size makes sense to me in
sectors.
What I will do instead is just to document this quirk. Look out for
V11 :)
--John
On 23.12.2014 04:12, John Snow wrote:
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
---
block.c | 39 +++
include/block/block.h | 4
2 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/block.c b/block.c
index a1d9e88..f9e0767 100644
--- a/block.c
+++ b/block.c
@@ -53,6 +53,8 @@
struct BdrvDirtyBitmap {
HBitmap *bitmap;
+int64_t size;
+int64_t granularity;
char *name;
QLIST_ENTRY(BdrvDirtyBitmap) list;
};
@@ -5343,6 +5345,21 @@ void
bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap)
bitmap->name = NULL;
}
+BdrvDirtyBitmap *bdrv_copy_dirty_bitmap(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap,
+const char *name)
+{
+BdrvDirtyBitmap *new_bitmap;
+
+new_bitmap = g_malloc0(sizeof(BdrvDirtyBitmap));
+new_bitmap->bitmap = hbitmap_copy(bitmap->bitmap);
+new_bitmap->size = bitmap->size;
+new_bitmap->granularity = bitmap->granularity;
+new_bitmap->name = g_strdup(name);
+QLIST_INSERT_HEAD(&bs->dirty_bitmaps, new_bitmap, list);
+return new_bitmap;
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
int granularity,
const char *name,
@@ -5350,6 +5367,7 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
{
int64_t bitmap_size;
BdrvDirtyBitmap *bitmap;
+int sector_granularity;
assert((granularity & (granularity - 1)) == 0);
@@ -5357,8 +5375,8 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
error_setg(errp, "Bitmap already exists: %s", name);
return NULL;
}
-granularity >>= BDRV_SECTOR_BITS;
-assert(granularity);
+sector_granularity = granularity >> BDRV_SECTOR_BITS;
+assert(sector_granularity);
bitmap_size = bdrv_nb_sectors(bs);
if (bitmap_size < 0) {
error_setg_errno(errp, -bitmap_size, "could not get length
of device");
@@ -5366,7 +5384,9 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(BlockDriverState *bs,
return NULL;
}
bitmap = g_new0(BdrvDirtyBitmap, 1);
-bitmap->bitmap = hbitmap_alloc(bitmap_size, ffs(granularity) - 1);
+bitmap->size = bitmap_size;
+bitmap->granularity = granularity;
+bitmap->bitmap = hbitmap_alloc(bitmap->size,
ffs(sector_granularity) - 1);
bitmap->name = g_strdup(name);
QLIST_INSERT_HEAD(&bs->dirty_bitmaps, bitmap, list);
return bitmap;
@@ -5439,7 +5459,9 @@ uint64_t
bdrv_get_default_bitmap_granularity(BlockDriverState *bs)
uint64_t bdrv_dirty_bitmap_granularity(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
{
-return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
+g_assert(BDRV_SECTOR_SIZE <<
hbitmap_granularity(bitmap->bitmap) ==
+ bitmap->granularity);
+return bitmap->granularity;
}
void bdrv_dirty_iter_init(BlockDriverState *bs,
@@ -5460,6 +5482,15 @@ void bdrv_reset_dirty_bitmap(BlockDriverState
*bs, BdrvDirtyBitmap *bitmap,
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
}
+/**
+ * Effectively, reset the hbitmap from bits [0, size)
+ * Synonymous with bdrv_reset_dirty_bitmap(bs, bitmap, 0,
bitmap->size)
+ */
+void bdrv_clear_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap
*bitmap)
+{
+hbitmap_reset(bitmap->bitmap, 0, bitmap->size);
+}
+
static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int nr_sectors)
{
diff --git a/include/block/block.h b/include/block/block.h
index c7402e7..e964abd 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -436,6 +436,9 @@ BdrvDirtyBitmap
*bdrv_create_dirty_bitmap(
Re: [Qemu-devel] [PATCH 4/9] hbitmap: store / restore
Best regards,
Vladimir
On 09.01.2015 00:21, John Snow wrote:
On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote:
Functions to store / restore HBitmap. HBitmap should be saved to linear
bitmap format independently of endianess.
Because of restoring in several steps, every step writes only the last
level of the bitmap. All other levels are restored by
hbitmap_restore_finish as a last step of restoring. So, HBitmap is
inconsistent while restoring.
.
I guess by nature of how bitmap migration has been implemented, we
cannot help but restore parts of the bitmap piece by piece, which
requires us to force the bitmap into an inconsistent state.
Yuck.
It might be "nice" to turn on a disable bit inside the hbitmap that
disallows its use until it is made consistent again, but I don't know
what the performance hit of the extra conditionals everywhere would be
like.
Another approach is to restore the bitmap after each piece set. It is a
bit slower of course but the bitmap will be consistent after
hbitmap_restore_data()
+/**
+ * hbitmap_restore_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_restore_data. Actuall all
HBitmap
+ * layers are restore here.
+ */
+void hbitmap_restore_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
These are just biased opinions:
- It might be nice to name the store/restore functions "serialize" and
"deserialize," to describe exactly what they are doing.
- I might refer to "restore_finish" as "post_load" or "post_restore"
or something similar to mimic how device migration functions are
named. I think hbitmap_restore_data_finalize would also be fine; the
key part here is clearly naming it relative to "restore_data."
Hmm. Ok, what about the following set:
hbitmap_serialize()
hbitmap_deserialize_part()
hbitmap_deserialize_finish()
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 8aa7406..ac0323f 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -362,6 +362,90 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] &
bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size;
+
+if (count == 0) {
+return 0;
+}
+
+size = (((count - 1) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
This seems flawed to me: number of bits without an offset can't be
mapped to a number of real bytes, because "two bits" may or may not
cross a granularity boundary, depending on *WHERE* you start counting.
e.g.
granularity = 1 (i.e. 2^1 = 2 virtual bits per 1 real bit)
virtual: 001100
real: 0 1 0
The amount of space required to hold "two bits" here could be as
little as one bit, if the offset is k={0,2,4} but it could be as much
as two bits if the offset is k={1,3}.
You may never use the function in this way, but mapping virtual bits
to an implementation byte-size seems like it is inviting an
inconsistency.
I dislike this function too.. But unfortunately we need the size in
bytes used for serialization.
Hmm. Ok, without loss of generality, let offset be less than
granularity. The precise formula should look like:
size = (((offset+count-1) >> hb->granularity) >> BITS_PER_LEVEL);
So,
size = 1 << hb->granularity) + count - 2) >> hb->granularity) >>
BITS_PER_LEVEL) + 1;
would be enough in any case. Ok?
+void hbitmap_store_data(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count)
+{
+uint64_t last = start + count - 1;
+unsigned long *out = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+for (i = start; i <= last; ++i) {
+unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+out[i] = (BITS_PER_LONG == 32 ? cpu_to_le32(el) :
cpu_to_le64(el));
+}
+#else
+memcpy(out, &hb->levels[HBITMAP_LEVELS - 1][start],
+ count * sizeof(unsigned long));
+#endif
+}
+
I suppose the ifdefs are trying to take an optimization by using
memcpy if at all possible, without a conversion.
Why are the conversions to little endian, though? Shouldn't we be
serializing to a Big Endian format?
As Paolo already explained it is for portability. LE bitmap
representation is independent of size of bitmap array elements, which
may be 32bit/64bit, or whatever else with LE format.
+void hbitmap_restore_data(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+
Re: [Qemu-devel] [PATCH v11 01/13] block: fix spoiling all dirty bitmaps by mirror and migration
it is already in master and can be dropped here
Best regards,
Vladimir
On 12.01.2015 19:30, John Snow wrote:
From: Vladimir Sementsov-Ogievskiy
Mirror and migration use dirty bitmaps for their purposes, and since
commit [block: per caller dirty bitmap] they use their own bitmaps, not
the global one. But they use old functions bdrv_set_dirty and
bdrv_reset_dirty, which change all dirty bitmaps.
Named dirty bitmaps series by Fam and Snow are affected: mirroring and
migration will spoil all (not related to this mirroring or migration)
named dirty bitmaps.
This patch fixes this by adding bdrv_set_dirty_bitmap and
bdrv_reset_dirty_bitmap, which change concrete bitmap. Also, to prevent
such mistakes in future, old functions bdrv_(set,reset)_dirty are made
static, for internal block usage.
Signed-off-by: Vladimir Sementsov-Ogievskiy
CC: John Snow
CC: Fam Zheng
CC: Denis V. Lunev
CC: Stefan Hajnoczi
CC: Kevin Wolf
Signed-off-by: John Snow
---
block.c | 23 ---
block/mirror.c| 11 +++
include/block/block.h | 6 --
migration/block.c | 5 +++--
4 files changed, 34 insertions(+), 11 deletions(-)
diff --git a/block.c b/block.c
index 4165d42..6c1b49a 100644
--- a/block.c
+++ b/block.c
@@ -97,6 +97,10 @@ static QTAILQ_HEAD(, BlockDriverState) graph_bdrv_states =
static QLIST_HEAD(, BlockDriver) bdrv_drivers =
QLIST_HEAD_INITIALIZER(bdrv_drivers);
+static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
+ int nr_sectors);
+static void bdrv_reset_dirty(BlockDriverState *bs, int64_t cur_sector,
+ int nr_sectors);
/* If non-zero, use only whitelisted block drivers */
static int use_bdrv_whitelist;
@@ -5389,8 +5393,20 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
hbitmap_iter_init(hbi, bitmap->bitmap, 0);
}
-void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
-int nr_sectors)
+void bdrv_set_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
+ int64_t cur_sector, int nr_sectors)
+{
+hbitmap_set(bitmap->bitmap, cur_sector, nr_sectors);
+}
+
+void bdrv_reset_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
+ int64_t cur_sector, int nr_sectors)
+{
+hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
+}
+
+static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
+ int nr_sectors)
{
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
@@ -5398,7 +5414,8 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t
cur_sector,
}
}
-void bdrv_reset_dirty(BlockDriverState *bs, int64_t cur_sector, int nr_sectors)
+static void bdrv_reset_dirty(BlockDriverState *bs, int64_t cur_sector,
+ int nr_sectors)
{
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
diff --git a/block/mirror.c b/block/mirror.c
index 2c6dd2a..9019d1b 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -128,7 +128,8 @@ static void mirror_write_complete(void *opaque, int ret)
BlockDriverState *source = s->common.bs;
BlockErrorAction action;
-bdrv_set_dirty(source, op->sector_num, op->nb_sectors);
+bdrv_set_dirty_bitmap(source, s->dirty_bitmap, op->sector_num,
+ op->nb_sectors);
action = mirror_error_action(s, false, -ret);
if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) {
s->ret = ret;
@@ -145,7 +146,8 @@ static void mirror_read_complete(void *opaque, int ret)
BlockDriverState *source = s->common.bs;
BlockErrorAction action;
-bdrv_set_dirty(source, op->sector_num, op->nb_sectors);
+bdrv_set_dirty_bitmap(source, s->dirty_bitmap, op->sector_num,
+ op->nb_sectors);
action = mirror_error_action(s, true, -ret);
if (action == BLOCK_ERROR_ACTION_REPORT && s->ret >= 0) {
s->ret = ret;
@@ -286,7 +288,8 @@ static uint64_t coroutine_fn
mirror_iteration(MirrorBlockJob *s)
next_sector += sectors_per_chunk;
}
-bdrv_reset_dirty(source, sector_num, nb_sectors);
+bdrv_reset_dirty_bitmap(source, s->dirty_bitmap, sector_num,
+nb_sectors);
/* Copy the dirty cluster. */
s->in_flight++;
@@ -442,7 +445,7 @@ static void coroutine_fn mirror_run(void *opaque)
assert(n > 0);
if (ret == 1) {
-bdrv_set_dirty(bs, sector_num, n);
+bdrv_set_dirty_bitmap(bs, s->dirty_bitmap, sector_num, n);
sector_num = next;
} else {
sector_num += n;
diff --git a/include/bloc
Re: [Qemu-devel] [PATCH v10 06/13] block: Add bdrv_copy_dirty_bitmap and bdrv_clear_dirty_bitmap
No, I'm saying about adding comment about using size in sectors and granularity in bytes in BdrvDirtyBitmap structure. On 10.01.2015 06:25, John Snow wrote: .. What I will do instead is just to document this quirk. Look out for V11 Best regards, Vladimir On 13.01.2015 19:37, John Snow wrote: On 01/13/2015 04:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hmm, can't find it in v11 Best regards, Vladimir I changed approach and instead of copying bitmaps to be used with the incremental backup, I lock them in place. So copy isn't used anymore in my set, so the name of the patch changed. --js
Re: [Qemu-devel] [PATCH v11 13/13] block: BdrvDirtyBitmap miscellaneous fixup
Hmm. May be, update the size field to be uint64_t too? Negative size is
not meaningful..
Best regards,
Vladimir
On 12.01.2015 19:31, John Snow wrote:
(1) granularity type consistency: Update the granularity to be uint64_t
in all places. This value is always in bytes.
(2) Some documentation for the fields within BdrvDirtyBitmap.
Signed-off-by: John Snow
---
block.c | 16
include/block/block.h | 2 +-
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/block.c b/block.c
index 9e2b8c0..13f9cc0 100644
--- a/block.c
+++ b/block.c
@@ -52,13 +52,13 @@
#endif
struct BdrvDirtyBitmap {
-HBitmap *bitmap;
-BdrvDirtyBitmap *successor;
-int64_t size;
-int64_t granularity;
-char *name;
-bool enabled;
-bool frozen;
+HBitmap *bitmap;/* Dirty sector bitmap */
+BdrvDirtyBitmap *successor; /* Anonymous child */
+int64_t size; /* Number of sectors */
+uint64_t granularity; /* Granularity in bytes */
+char *name; /* Optional non-empty unique ID */
+bool enabled; /* Writes are being tracked */
+bool frozen;/* Protected; see bdrv_freeze_dirty_bitmap() */
QLIST_ENTRY(BdrvDirtyBitmap) list;
};
@@ -5350,7 +5350,7 @@ void bdrv_dirty_bitmap_make_anon(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
}
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
- int granularity,
+ uint64_t granularity,
const char *name,
Error **errp)
{
diff --git a/include/block/block.h b/include/block/block.h
index 99ed679..749429e 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -430,7 +430,7 @@ bool bdrv_qiov_is_aligned(BlockDriverState *bs,
QEMUIOVector *qiov);
struct HBitmapIter;
typedef struct BdrvDirtyBitmap BdrvDirtyBitmap;
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
- int granularity,
+ uint64_t granularity,
const char *name,
Error **errp);
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
[Qemu-devel] [PATCH 0/8] block: persistent dirty bitmaps (RFC)
The bitmaps are saved into qcow2 file format. It provides both 'internal' and 'external' dirty bitmaps feature: - for qcow2 drives we can store bitmaps in the same file - for other formats we can store bitmaps in the separate qcow2 file QCow2 header is extended by fields 'nb_dirty_bitmaps' and 'dirty_bitmaps_offset' like with snapshots. Proposed command line syntax is the following: -dirty-bitmap [option1=val1][,option2=val2]... Available options are: name The name for the bitmap (necessary). file The file to load the bitmap from. file_id When specified with 'file' option, then this file will be available through this id for other -dirty-bitmap options when specified without 'file' option, then it is a reference to 'file', specified with another -dirty-bitmap option, and it will be used to load the bitmap from. driveThe drive to bind the bitmap to. It should be specified as 'id' suboption of one of -drive options. If nor 'file' neither 'file_id' are specified, then the bitmap will be loaded from that drive (internal dirty bitmap). granularity The granularity for the bitmap. Not necessary, the default value may be used. enabled on|off. Default is 'on'. Disabled bitmaps are not changing regardless of writes to corresponding drive. Examples: qemu -drive file=a.qcow2,id=disk -dirty-bitmap name=b,drive=disk qemu -drive file=a.raw,id=disk \ -dirty-bitmap name=b,drive=disk,file=b.qcow2,enabled=off Vladimir Sementsov-Ogievskiy (8): spec: add qcow2-dirty-bitmaps specification hbitmap: store / restore qcow2: add dirty-bitmaps feature block: store persistent dirty bitmaps block: add bdrv_load_dirty_bitmap qemu: command line option for dirty bitmaps qmp: print dirty bitmap iotests: test internal persistent dirty bitmap block.c| 113 ++ block/Makefile.objs| 2 +- block/qcow2-dirty-bitmap.c | 514 + block/qcow2.c | 26 +++ block/qcow2.h | 48 + blockdev.c | 51 + docs/specs/qcow2.txt | 59 ++ hmp-commands.hx| 15 ++ hmp.c | 8 + hmp.h | 1 + include/block/block.h | 9 + include/block/block_int.h | 10 + include/qemu/hbitmap.h | 49 + include/sysemu/blockdev.h | 1 + include/sysemu/sysemu.h| 1 + qapi-schema.json | 3 +- qapi/block-core.json | 3 + qemu-options.hx| 37 qmp-commands.hx| 5 + tests/qemu-iotests/115 | 96 + tests/qemu-iotests/115.out | 64 ++ tests/qemu-iotests/group | 1 + util/hbitmap.c | 87 vl.c | 100 + 24 files changed, 1301 insertions(+), 2 deletions(-) create mode 100644 block/qcow2-dirty-bitmap.c create mode 100755 tests/qemu-iotests/115 create mode 100644 tests/qemu-iotests/115.out -- 1.9.1
[Qemu-devel] [PATCH 8/8] iotests: test internal persistent dirty bitmap
The test performs several vm reloads with checking and updating dirty
bitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/115 | 96 ++
tests/qemu-iotests/115.out | 64 +++
tests/qemu-iotests/group | 1 +
3 files changed, 161 insertions(+)
create mode 100755 tests/qemu-iotests/115
create mode 100644 tests/qemu-iotests/115.out
diff --git a/tests/qemu-iotests/115 b/tests/qemu-iotests/115
new file mode 100755
index 000..ef853b1
--- /dev/null
+++ b/tests/qemu-iotests/115
@@ -0,0 +1,96 @@
+#!/bin/bash
+#
+# Persistent dirty bitmap test
+#
+# Performs several vm reloads with checking and updating dirty bitmap.
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=vsement...@parallels.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1# failure is the default!
+
+_cleanup()
+{
+_cleanup_qemu
+_cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.qemu
+
+_supported_fmt qcow2
+_supported_os Linux
+
+size=1G
+_make_test_img $size
+
+qemu_comm_method="monitor"
+
+launch() {
+_launch_qemu -drive file="${TEST_IMG}",id=disk\
+-dirty-bitmap name=bitmap,drive=disk
+echo
+echo START VM
+}
+
+print_bitmap() {
+_send_qemu_cmd $QEMU_HANDLE 'print_dirty_bitmap disk bitmap'\
+"dirty regions end" | sed 's/^//'
+}
+
+write() {
+_send_qemu_cmd $QEMU_HANDLE 'qemu-io disk "write ' $@ '"' "(qemu)"
+echo
+echo write $@
+}
+
+quit() {
+_send_qemu_cmd $QEMU_HANDLE 'qemu-io disk flush' "(qemu)"
+_send_qemu_cmd $QEMU_HANDLE 'quit' ""
+echo QUIT
+wait ${QEMU_PID[$QEMU_HANDLE]} 2>/dev/null
+}
+
+{
+launch
+print_bitmap
+write 50m 1m
+print_bitmap
+quit
+
+launch
+print_bitmap
+write 0m 10m
+write 700m 200m
+print_bitmap
+quit
+
+launch
+print_bitmap
+quit
+} | sed '/(qemu)/d'
+
+status=0
diff --git a/tests/qemu-iotests/115.out b/tests/qemu-iotests/115.out
new file mode 100644
index 000..d570c9f
--- /dev/null
+++ b/tests/qemu-iotests/115.out
@@ -0,0 +1,64 @@
+QA output created by 115
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
+
+START VM
+QEMU X.Y.Z monitor - type 'help' for more information
+bitmap 'bitmap'
+enabled: true
+size: 2097152
+granularity: 65536
+dirty regions begin:
+dirty regions end
+
+write 50m 1m
+wrote 1048576/1048576 bytes at offset 52428800
+1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+bitmap 'bitmap'
+enabled: true
+size: 2097152
+granularity: 65536
+dirty regions begin:
+102400 -> 104447
+dirty regions end
+QUIT
+
+START VM
+QEMU X.Y.Z monitor - type 'help' for more information
+bitmap 'bitmap'
+enabled: true
+size: 2097152
+granularity: 65536
+dirty regions begin:
+102400 -> 104447
+dirty regions end
+
+write 0m 10m
+wrote 10485760/10485760 bytes at offset 0
+10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+write 700m 200m
+wrote 209715200/209715200 bytes at offset 734003200
+200 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+bitmap 'bitmap'
+enabled: true
+size: 2097152
+granularity: 65536
+dirty regions begin:
+0 -> 20479
+102400 -> 104447
+1433600 -> 1843199
+dirty regions end
+QUIT
+
+START VM
+QEMU X.Y.Z monitor - type 'help' for more information
+bitmap 'bitmap'
+enabled: true
+size: 2097152
+granularity: 65536
+dirty regions begin:
+0 -> 20479
+102400 -> 104447
+1433600 -> 1843199
+dirty regions end
+QUIT
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index a4742c6..77377b3 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -115,3 +115,4 @@
111 rw auto quick
113 rw auto quick
114 rw auto quick
+115 rw auto quick
--
1.9.1
[Qemu-devel] [PATCH 2/8] hbitmap: store / restore
Functions to store / restore HBitmap. HBitmap should be saved to linear
bitmap format independently of endianess.
These functions are appropriate for dirty bitmap migration, retoring the
bitmap in several steps is available. To save performance, every step
writes only the last level of the bitmap. All other levels are restored
by hbitmap_restore_finish as a last step of restoring. So, HBitmap is
inconsistent while restoring.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/qemu/hbitmap.h | 49
util/hbitmap.c | 87 ++
2 files changed, 136 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index c48c50a..f432f7f 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -137,6 +137,55 @@ void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t
count);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_data_size:
+ * @hb: HBitmap to operate on.
+ * @count: Number of bits
+ *
+ * Return amount of bytes hbitmap_store_data needs
+ */
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count);
+
+/**
+ * hbitmap_store_data
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to store bitmap data.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_restore_data
+ * independently of endianess
+ */
+void hbitmap_store_data(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_restore_data
+ * @hb: HBitmap to oprate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ *
+ * Retores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_store_data.
+ *
+ * ! The bitmap becomes inconsistent after this operation.
+ * hbitmap_restore_finish should be called before using the bitmap after
+ * data restoring.
+ */
+void hbitmap_restore_data(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_restore_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_restore_data. Actuall all HBitmap
+ * layers are restore here.
+ */
+void hbitmap_restore_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index f400dcb..5d1a776 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -366,6 +366,93 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size;
+
+if (count == 0) {
+return 0;
+}
+
+size = (((count - 1) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
+void hbitmap_store_data(const HBitmap *hb, uint8_t *buf,
+uint64_t start, uint64_t count)
+{
+uint64_t last = start + count - 1;
+unsigned long *out = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+for (i = start; i <= last; ++i) {
+unsigned long el = hb->levels[HBITMAP_LEVELS - 1][i];
+out[i] = (BITS_PER_LONG == 32 ? cpu_to_le32(el) : cpu_to_le64(el));
+}
+#else
+memcpy(out, &hb->levels[HBITMAP_LEVELS - 1][start],
+ count * sizeof(unsigned long));
+#endif
+}
+
+void hbitmap_restore_data(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+uint64_t last = start + count - 1;
+unsigned long *in = (unsigned long *)buf;
+
+if (count == 0) {
+return;
+}
+
+start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+count = last - start + 1;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+for (i = start; i <= last; ++i) {
+hb->levels[HBITMAP_LEVELS - 1][i] =
+(BITS_PER_LONG == 32 ? be32_to_cpu(in[i]) : be64_to_cpu(in[i]));
+}
+#else
+memcpy(&hb->levels[HBITMAP_LEVELS - 1][start], in,
+ count * sizeof(unsigned long));
+#endif
+}
+
+void hbitmap_restore_finish(HBitmap *bitmap)
+{
+int64_t i, size, prev_size;
+int lev;
+
+/* restore levels starting from penultimate to zero level, assuming
+ * that the last level is ok */
+size = MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+for (lev = HBITMAP_LEVELS - 1; lev-- > 0; ) {
+prev_size = size;
+si
[Qemu-devel] [PATCH 1/8] spec: add qcow2-dirty-bitmaps specification
Persistent dirty bitmaps will be saved into qcow2 files. It may be used as 'internal' bitmaps (for qcow2 drives) or as 'external' bitmaps for other drives (there may be qcow2 file with zero disk size but with several dirty bitmaps for other drives). Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/specs/qcow2.txt | 59 1 file changed, 59 insertions(+) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 121dfc8..b29de40 100644 --- a/docs/specs/qcow2.txt +++ b/docs/specs/qcow2.txt @@ -116,6 +116,13 @@ in the description of a field. Length of the header structure in bytes. For version 2 images, the length is always assumed to be 72 bytes. +104 - 107: nb_dirty_bitmaps +Number of dirty bitmaps contained in the image + +108 - 115: dirty_bitmaps_offset +Offset into the image file at which the dirty bitmaps table +starts. Must be aligned to a cluster boundary. + Directly after the image header, optional sections called header extensions can be stored. Each extension has a structure like the following: @@ -360,3 +367,55 @@ Snapshot table entry: variable: Padding to round up the snapshot table entry size to the next multiple of 8. + + +== Dirty bitmaps == + +The feature supports storing several dirty bitmaps in the qcow2 file. + +=== Cluster mapping === + +Dirty bitmaps are stored using a ONE-level structure for the mapping of +bitmaps to host clusters. There only an L1 table. + +The L1 table has a variable size (stored in the Bitmap table entry) and may +use multiple clusters, however it must be contiguous in the image file. + +Given an offset into the bitmap, the offset into the image file can be +obtained as follows: + +offset = l1_table[offset / cluster_size] + (offset % cluster_size) + +L1 table entry: + +Bit 0 - 61: Standard cluster descriptor + +62 - 63: Reserved + +=== Bitmap table === + +A directory of all bitmaps is stored in the bitmap table, a contiguous area in +the image file, whose starting offset and length are given by the header fields +dirty_bitmaps_offset and nb_dirty_bitmaps. The entries of the bitmap table have +variable length, depending on the length of name and extra data. + +Bitmap table entry: + +Byte 0 - 7:Offset into the image file at which the L1 table for the +bitmap starts. Must be aligned to a cluster boundary. + + 8 - 11:Number of entries in the L1 table of the bitmap + +12 - 15:Bitmap granularity in bytes + +16 - 23:Bitmap size in sectors + +24 - 25:Size of the bitmap name + +variable: The name of the bitmap (not null terminated) + +variable: Padding to round up the bitmap table entry size to the +next multiple of 8. + +The fields "size", "granularity" and "name" are corresponding with the fields +in struct BdrvDirtyBitmap. -- 1.9.1
[Qemu-devel] [PATCH 6/8] qemu: command line option for dirty bitmaps
The patch adds the following command line option:
-dirty-bitmap [option1=val1][,option2=val2]...
Available options are:
name The name for the bitmap (necessary).
file The file to load the bitmap from.
file_id When specified with 'file' option, then this file will
be available through this id for other -dirty-bitmap
options when specified without 'file' option, then it
is a reference to 'file', specified with another
-dirty-bitmap option, and it will be used to load the
bitmap from.
driveThe drive to bind the bitmap to. It should be specified
as 'id' suboption of one of -drive options. If nor
'file' neither 'file_id' are specified, then the bitmap
will be loaded from that drive (internal dirty bitmap).
granularity The granularity for the bitmap. Not necessary, the
default value may be used.
enabled on|off. Default is 'on'. Disabled bitmaps are not
changing regardless of writes to corresponding drive.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
blockdev.c| 38 ++
include/sysemu/blockdev.h | 1 +
include/sysemu/sysemu.h | 1 +
qemu-options.hx | 37 +
vl.c | 100 ++
5 files changed, 177 insertions(+)
diff --git a/blockdev.c b/blockdev.c
index 209fedd..8a9be08 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -176,6 +176,11 @@ QemuOpts *drive_def(const char *optstr)
return qemu_opts_parse(qemu_find_opts("drive"), optstr, 0);
}
+QemuOpts *dirty_bitmap_def(const char *optstr)
+{
+return qemu_opts_parse(qemu_find_opts("dirty-bitmap"), optstr, 0);
+}
+
QemuOpts *drive_add(BlockInterfaceType type, int index, const char *file,
const char *optstr)
{
@@ -3044,6 +3049,39 @@ BlockJobInfoList *qmp_query_block_jobs(Error **errp)
return head;
}
+QemuOptsList qemu_dirty_bitmap_opts = {
+.name = "dirty-bitmap",
+.head = QTAILQ_HEAD_INITIALIZER(qemu_dirty_bitmap_opts.head),
+.desc = {
+{
+.name = "name",
+.type = QEMU_OPT_STRING,
+.help = "Name of the dirty bitmap",
+},{
+.name = "file",
+.type = QEMU_OPT_STRING,
+.help = "file name to load the bitmap from",
+},{
+.name = "file_id",
+.type = QEMU_OPT_STRING,
+.help = "node name to load the bitmap from (or to set id for"
+" for file, opened by previous option)",
+},{
+.name = "drive",
+.type = QEMU_OPT_STRING,
+.help = "drive id to bind the bitmap to",
+},{
+.name = "granularity",
+.type = QEMU_OPT_NUMBER,
+.help = "granularity",
+},{
+.name = "enabled",
+.type = QEMU_OPT_BOOL,
+.help = "enabled flag (default is 'on')",
+}
+}
+};
+
QemuOptsList qemu_common_drive_opts = {
.name = "drive",
.head = QTAILQ_HEAD_INITIALIZER(qemu_common_drive_opts.head),
diff --git a/include/sysemu/blockdev.h b/include/sysemu/blockdev.h
index 09d1e30..48cce5c 100644
--- a/include/sysemu/blockdev.h
+++ b/include/sysemu/blockdev.h
@@ -57,6 +57,7 @@ int drive_get_max_devs(BlockInterfaceType type);
DriveInfo *drive_get_next(BlockInterfaceType type);
QemuOpts *drive_def(const char *optstr);
+QemuOpts *dirty_bitmap_def(const char *optstr);
QemuOpts *drive_add(BlockInterfaceType type, int index, const char *file,
const char *optstr);
DriveInfo *drive_new(QemuOpts *arg, BlockInterfaceType block_default_type);
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 503e5a4..c984a52 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -231,6 +231,7 @@ bool usb_enabled(bool default_usb);
extern QemuOptsList qemu_legacy_drive_opts;
extern QemuOptsList qemu_common_drive_opts;
+extern QemuOptsList qemu_dirty_bitmap_opts;
extern QemuOptsList qemu_drive_opts;
extern QemuOptsList qemu_chardev_opts;
extern QemuOptsList qemu_device_opts;
diff --git a/qemu-options.hx b/qemu-options.hx
index 10b9568..3a5bfde 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -592,6 +592,43 @@ qemu-system-i386 -hda a -hdb b
@end example
ETEXI
+DEF("dirty-bitmap", HAS_ARG, QEMU_OPTION_dirty_bitmap,
+"-dirty-bitmap name=name[,file=file][,file_id=file_id][,drive=@var{id}]\n"
+" [,granularity=granularity][,enabled=on|off]\n",
+QEMU_ARCH_ALL)
+STEXI
[Qemu-devel] [PATCH 4/8] block: store persistent dirty bitmaps
Persistent dirty bitmaps are the bitmaps, for which the new field
BdrvDirtyBitmap.file is not NULL. We save all persistent dirty bitmaps
owned by BlockDriverState in corresponding bdrv_close().
BdrvDirtyBitmap.file is a BlockDriverState, where we want to save the
bitmap. It may be set in bdrv_dirty_bitmap_set_file() only once.
bdrv_ref/bdrv_unref are used for BdrvDirtyBitmap.file to be sure that
files will be closed and resources will be freed.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 43 +++
include/block/block.h | 3 +++
2 files changed, 46 insertions(+)
diff --git a/block.c b/block.c
index 2466ba8..7237b95 100644
--- a/block.c
+++ b/block.c
@@ -54,6 +54,7 @@
struct BdrvDirtyBitmap {
HBitmap *bitmap;
BdrvDirtyBitmap *originator;
+BlockDriverState *file;
int64_t size;
int64_t granularity;
char *name;
@@ -1840,6 +1841,7 @@ void bdrv_reopen_abort(BDRVReopenState *reopen_state)
void bdrv_close(BlockDriverState *bs)
{
BdrvAioNotifier *ban, *ban_next;
+BdrvDirtyBitmap *bm, *bm_next;
if (bs->job) {
block_job_cancel_sync(bs->job);
@@ -1849,6 +1851,15 @@ void bdrv_close(BlockDriverState *bs)
bdrv_drain_all(); /* in case flush left pending I/O */
notifier_list_notify(&bs->close_notifiers, bs);
+/* save and release persistent dirty bitmaps */
+QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, bm_next) {
+if (bm->file) {
+bdrv_store_dirty_bitmap(bm);
+bdrv_unref(bm->file);
+bdrv_release_dirty_bitmap(bs, bm);
+}
+}
+
if (bs->drv) {
if (bs->backing_hd) {
BlockDriverState *backing_hd = bs->backing_hd;
@@ -5373,6 +5384,29 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
return originator;
}
+int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+BlockDriverState *bs = bitmap->file;
+uint8_t *buf;
+uint64_t size;
+assert(bs);
+assert(bs->drv);
+assert(bs->drv->bdrv_dirty_bitmap_store);
+
+size = hbitmap_data_size(bitmap->bitmap, bitmap->size);
+size = (size + 3) & ~3;
+buf = g_malloc(size);
+
+hbitmap_store_data(bitmap->bitmap, buf, 0, bitmap->size);
+
+int res = bs->drv->bdrv_dirty_bitmap_store(bs, buf,
+ bitmap->name,
+ bitmap->size,
+ bitmap->granularity);
+
+g_free(buf);
+return res;
+}
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
int granularity,
@@ -5421,6 +5455,15 @@ void bdrv_release_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap)
}
}
+void bdrv_dirty_bitmap_set_file(BdrvDirtyBitmap *bitmap, BlockDriverState
*file)
+{
+assert(bitmap->file == NULL);
+bitmap->file = file;
+if (file != NULL) {
+bdrv_ref(file);
+}
+}
+
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
bitmap->enabled = false;
diff --git a/include/block/block.h b/include/block/block.h
index cb1f28d..0dfefe3 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -442,6 +442,8 @@ BdrvDirtyBitmap *bdrv_copy_dirty_bitmap(BlockDriverState
*bs,
BdrvDirtyBitmap *bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *failed);
void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+void bdrv_dirty_bitmap_set_file(BdrvDirtyBitmap *bitmap,
+BlockDriverState *file);
void bdrv_disable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
@@ -458,6 +460,7 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, struct HBitmapIter *hbi);
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH 3/8] qcow2: add dirty-bitmaps feature
Adds dirty-bitmaps feature to qcow2 format as specified in
docs/specs/qcow2.txt
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/Makefile.objs| 2 +-
block/qcow2-dirty-bitmap.c | 514 +
block/qcow2.c | 26 +++
block/qcow2.h | 48 +
include/block/block_int.h | 10 +
5 files changed, 599 insertions(+), 1 deletion(-)
create mode 100644 block/qcow2-dirty-bitmap.c
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 04b0e43..eebd1c9 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -1,5 +1,5 @@
block-obj-y += raw_bsd.o qcow.o vdi.o vmdk.o cloop.o dmg.o bochs.o vpc.o
vvfat.o
-block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o
qcow2-cache.o
+block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o
qcow2-cache.o qcow2-dirty-bitmap.o
block-obj-y += qed.o qed-gencb.o qed-l2-cache.o qed-table.o qed-cluster.o
block-obj-y += qed-check.o
block-obj-$(CONFIG_VHDX) += vhdx.o vhdx-endian.o vhdx-log.o
diff --git a/block/qcow2-dirty-bitmap.c b/block/qcow2-dirty-bitmap.c
new file mode 100644
index 000..b3d114f
--- /dev/null
+++ b/block/qcow2-dirty-bitmap.c
@@ -0,0 +1,514 @@
+/*
+ * Dirty bitmpas for the QCOW version 2 format
+ *
+ * Copyright (c) 2014-2015 Vladimir Sementsov-Ogievskiy
+ *
+ * This file is derived from qcow2-snapshot.c, original copyright:
+ * Copyright (c) 2004-2006 Fabrice Bellard
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu-common.h"
+#include "block/block_int.h"
+#include "block/qcow2.h"
+
+void qcow2_free_dirty_bitmaps(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaque;
+int i;
+
+for (i = 0; i < s->nb_dirty_bitmaps; i++) {
+g_free(s->dirty_bitmaps[i].name);
+}
+g_free(s->dirty_bitmaps);
+s->dirty_bitmaps = NULL;
+s->nb_dirty_bitmaps = 0;
+}
+
+int qcow2_read_dirty_bitmaps(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaque;
+QCowDirtyBitmapHeader h;
+QCowDirtyBitmap *bm;
+int i, name_size;
+int64_t offset;
+int ret;
+
+if (!s->nb_dirty_bitmaps) {
+s->dirty_bitmaps = NULL;
+s->dirty_bitmaps_size = 0;
+return 0;
+}
+
+offset = s->dirty_bitmaps_offset;
+s->dirty_bitmaps = g_new0(QCowDirtyBitmap, s->nb_dirty_bitmaps);
+
+for (i = 0; i < s->nb_dirty_bitmaps; i++) {
+/* Read statically sized part of the dirty_bitmap header */
+offset = align_offset(offset, 8);
+ret = bdrv_pread(bs->file, offset, &h, sizeof(h));
+if (ret < 0) {
+goto fail;
+}
+
+offset += sizeof(h);
+bm = s->dirty_bitmaps + i;
+bm->l1_table_offset = be64_to_cpu(h.l1_table_offset);
+bm->l1_size = be32_to_cpu(h.l1_size);
+bm->bitmap_granularity = be32_to_cpu(h.bitmap_granularity);
+bm->bitmap_size = be64_to_cpu(h.bitmap_size);
+
+name_size = be16_to_cpu(h.name_size);
+
+/* Read dirty_bitmap name */
+bm->name = g_malloc(name_size + 1);
+ret = bdrv_pread(bs->file, offset, bm->name, name_size);
+if (ret < 0) {
+goto fail;
+}
+offset += name_size;
+bm->name[name_size] = '\0';
+
+if (offset - s->dirty_bitmaps_offset > QCOW_MAX_DIRTY_BITMAPS_SIZE) {
+ret = -EFBIG;
+goto fail;
+}
+}
+
+assert(offset - s->dirty_bitmaps_offset <= INT_MAX);
+s->dirty_bitmaps_size = offset - s->dirty_bitmaps_offset;
+return 0;
+
+fail:
+qcow2_free_dirty_bitmaps(bs);
+return ret;
+}
+
+/* add at the end of the file a new list of dirty bitmaps */
+static int qcow2_write_dirty_bitmaps(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaqu
[Qemu-devel] [PATCH 5/8] block: add bdrv_load_dirty_bitmap
The funcion loads dirty bitmap from file, using underlying driver
function.
Note: the function doesn't change BdrvDirtyBitmap.file field. This field
is only used by bdrv_store_dirty_bitmap() function and is ONLY written
by bdrv_dirty_bitmap_set_file() function.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 37 +
include/block/block.h | 5 +
2 files changed, 42 insertions(+)
diff --git a/block.c b/block.c
index 7237b95..77419e9 100644
--- a/block.c
+++ b/block.c
@@ -5384,6 +5384,43 @@ BdrvDirtyBitmap
*bdrv_reclaim_dirty_bitmap(BlockDriverState *bs,
return originator;
}
+BdrvDirtyBitmap *bdrv_load_dirty_bitmap(BlockDriverState *bs,
+BlockDriverState *file,
+int granularity,
+const char *name,
+Error **errp)
+{
+BlockDriver *drv = file->drv;
+if (!drv) {
+return NULL;
+}
+if (drv->bdrv_dirty_bitmap_load) {
+BdrvDirtyBitmap *bitmap;
+uint64_t bitmap_size = bdrv_nb_sectors(bs);
+uint8_t *buf = drv->bdrv_dirty_bitmap_load(file, name, bitmap_size,
+ granularity);
+if (buf == NULL) {
+return NULL;
+}
+
+bitmap = bdrv_create_dirty_bitmap(bs, granularity, name, errp);
+if (bitmap == NULL) {
+g_free(buf);
+return NULL;
+}
+
+hbitmap_restore_data(bitmap->bitmap, buf, 0, bitmap_size);
+hbitmap_restore_finish(bitmap->bitmap);
+
+return bitmap;
+}
+if (file->file) {
+return bdrv_load_dirty_bitmap(bs, file->file, granularity, name,
+ errp);
+}
+return NULL;
+}
+
int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap)
{
BlockDriverState *bs = bitmap->file;
diff --git a/include/block/block.h b/include/block/block.h
index 0dfefe3..f36557f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -460,6 +460,11 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, struct HBitmapIter *hbi);
void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap *bdrv_load_dirty_bitmap(BlockDriverState *bs,
+BlockDriverState *file,
+int granularity,
+const char *name,
+Error **errp);
int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_copy_on_read(BlockDriverState *bs);
--
1.9.1
[Qemu-devel] [PATCH 7/8] qmp: print dirty bitmap
Adds qmp and hmp commands to print dirty bitmap. This is needed only for
testing persistent dirty bitmap feature.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 33 +
blockdev.c| 13 +
hmp-commands.hx | 15 +++
hmp.c | 8
hmp.h | 1 +
include/block/block.h | 1 +
qapi-schema.json | 3 ++-
qapi/block-core.json | 3 +++
qmp-commands.hx | 5 +
9 files changed, 81 insertions(+), 1 deletion(-)
diff --git a/block.c b/block.c
index 77419e9..3e6dedf 100644
--- a/block.c
+++ b/block.c
@@ -5445,6 +5445,39 @@ int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap)
return res;
}
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+unsigned long a = 0, b = 0;
+
+printf("bitmap '%s'\n", bitmap->name ? bitmap->name : "no name");
+printf("enabled: %s\n", bitmap->enabled ? "true" : "false");
+printf("size: %" PRId64 "\n", bitmap->size);
+printf("granularity: %" PRId64 "\n", bitmap->granularity);
+printf("dirty regions begin:\n");
+
+while (true) {
+for (a = b; a < bitmap->size && !hbitmap_get(bitmap->bitmap, a); ++a) {
+;
+}
+if (a >= bitmap->size) {
+break;
+}
+
+for (b = a + 1;
+ b < bitmap->size && hbitmap_get(bitmap->bitmap, b);
+ ++b) {
+;
+}
+
+printf("%ld -> %ld\n", a, b - 1);
+if (b >= bitmap->size) {
+break;
+}
+}
+
+printf("dirty regions end\n");
+}
+
BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
int granularity,
const char *name,
diff --git a/blockdev.c b/blockdev.c
index 8a9be08..8b58c2e 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2079,6 +2079,19 @@ void qmp_block_dirty_bitmap_add(const char *node_ref,
const char *name,
aio_context_release(aio_context);
}
+void qmp_block_dirty_bitmap_print(const char *node_ref, const char *name,
+ Error **errp)
+{
+BdrvDirtyBitmap *bitmap;
+
+bitmap = block_dirty_bitmap_lookup(node_ref, name, NULL, errp);
+if (!bitmap) {
+return;
+}
+
+bdrv_print_dirty_bitmap(bitmap);
+}
+
void qmp_block_dirty_bitmap_remove(const char *node_ref, const char *name,
Error **errp)
{
diff --git a/hmp-commands.hx b/hmp-commands.hx
index e37bc8b..a9be506 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -58,6 +58,21 @@ Quit the emulator.
ETEXI
{
+.name = "print_dirty_bitmap",
+.args_type = "device:B,bitmap:s",
+.params = "device bitmap",
+.help = "print dirty bitmap",
+.user_print = monitor_user_noop,
+.mhandler.cmd = hmp_print_dirty_bitmap,
+},
+
+STEXI
+@item print_dirty_bitmap device_id bitmap_name
+@findex print_dirty_bitmap
+Print dirty bitmap meta information and dirty regions.
+ETEXI
+
+{
.name = "block_resize",
.args_type = "device:B,size:o",
.params = "device size",
diff --git a/hmp.c b/hmp.c
index 63b19c7..a269145 100644
--- a/hmp.c
+++ b/hmp.c
@@ -782,6 +782,14 @@ void hmp_info_tpm(Monitor *mon, const QDict *qdict)
qapi_free_TPMInfoList(info_list);
}
+void hmp_print_dirty_bitmap(Monitor *mon, const QDict *qdict)
+{
+const char *device = qdict_get_str(qdict, "device");
+const char *name = qdict_get_str(qdict, "bitmap");
+
+qmp_block_dirty_bitmap_print(device, name, NULL);
+}
+
void hmp_quit(Monitor *mon, const QDict *qdict)
{
monitor_suspend(mon);
diff --git a/hmp.h b/hmp.h
index 4bb5dca..6bbbc33 100644
--- a/hmp.h
+++ b/hmp.h
@@ -19,6 +19,7 @@
#include "qapi-types.h"
#include "qapi/qmp/qdict.h"
+void hmp_print_dirty_bitmap(Monitor *mon, const QDict *qdict);
void hmp_info_name(Monitor *mon, const QDict *qdict);
void hmp_info_version(Monitor *mon, const QDict *qdict);
void hmp_info_kvm(Monitor *mon, const QDict *qdict);
diff --git a/include/block/block.h b/include/block/block.h
index f36557f..7188791 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -466,6 +466,7 @@ BdrvDirtyBitmap *bdrv_load_dirty_bitmap(BlockDriverState
*bs,
const char *name,
Error **errp);
int bdrv_store_dirty_bitmap(BdrvDirtyBitmap *bitmap);
+void bdrv_print_dirty_bitmap(BdrvDirtyBitmap *bitmap);
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
diff
Re: [Qemu-devel] [PATCH 4/9] hbitmap: store / restore
Best regards,
Vladimir
On 13.01.2015 20:08, John Snow wrote:
On 01/13/2015 07:59 AM, Vladimir Sementsov-Ogievskiy wrote:
On 09.01.2015 00:21, John Snow wrote:
On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote:
+/**
+ * hbitmap_restore_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_restore_data. Actuall all
HBitmap
+ * layers are restore here.
+ */
+void hbitmap_restore_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
These are just biased opinions:
- It might be nice to name the store/restore functions "serialize" and
"deserialize," to describe exactly what they are doing.
- I might refer to "restore_finish" as "post_load" or "post_restore"
or something similar to mimic how device migration functions are
named. I think hbitmap_restore_data_finalize would also be fine; the
key part here is clearly naming it relative to "restore_data."
Hmm. Ok, what about the following set:
hbitmap_serialize()
hbitmap_deserialize_part()
hbitmap_deserialize_finish()
Looks good to me!
* hbitmap_serialize_part() is better to be similar with its pair function
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 8aa7406..ac0323f 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -362,6 +362,90 @@ bool hbitmap_get(const HBitmap *hb, uint64_t
item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] &
bit) != 0;
}
+uint64_t hbitmap_data_size(const HBitmap *hb, uint64_t count)
+{
+uint64_t size;
+
+if (count == 0) {
+return 0;
+}
+
+size = (((count - 1) >> hb->granularity) >> BITS_PER_LEVEL) + 1;
+
+return size * sizeof(unsigned long);
+}
+
This seems flawed to me: number of bits without an offset can't be
mapped to a number of real bytes, because "two bits" may or may not
cross a granularity boundary, depending on *WHERE* you start counting.
e.g.
granularity = 1 (i.e. 2^1 = 2 virtual bits per 1 real bit)
virtual: 001100
real: 0 1 0
The amount of space required to hold "two bits" here could be as
little as one bit, if the offset is k={0,2,4} but it could be as much
as two bits if the offset is k={1,3}.
You may never use the function in this way, but mapping virtual bits
to an implementation byte-size seems like it is inviting an
inconsistency.
I dislike this function too.. But unfortunately we need the size in
bytes used for serialization.
Hmm. Ok, without loss of generality, let offset be less than
granularity. The precise formula should look like:
size = (((offset+count-1) >> hb->granularity) >> BITS_PER_LEVEL);
So,
size = 1 << hb->granularity) + count - 2) >> hb->granularity) >>
BITS_PER_LEVEL) + 1;
would be enough in any case. Ok?
I think so, as long as when you deserialize the object it does so
correctly, regardless of whether or not you start on an even multiple
of the granularity.
May be, also rename hbitmap_data_size to hbitmap_serialize_size or even
drop it and make function hbitmap_store_data() return the necessary size
when *buf is NULL.
Re: [Qemu-devel] [PATCH 5/9] block: BdrvDirtyBitmap store/restore interface
As in previous patch, rename store/restore to serialize/deserialize...
Hmm. In this case, isn't it be better to include serialization of
granularity, name and name length in these functions?
Best regards,
Vladimir
On 11.12.2014 17:17, Vladimir Sementsov-Ogievskiy wrote:
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block.c | 61 +--
include/block/block.h | 10 +
2 files changed, 69 insertions(+), 2 deletions(-)
diff --git a/block.c b/block.c
index 6edf1dc..7d42620 100644
--- a/block.c
+++ b/block.c
@@ -5511,8 +5511,65 @@ void bdrv_reset_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
}
-static void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
- int nr_sectors)
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+return bitmap->name;
+}
+
+uint64_t bdrv_dbm_data_size(const BdrvDirtyBitmap *bitmap, uint64_t count)
+{
+return hbitmap_data_size(bitmap->bitmap, count);
+}
+
+void bdrv_dbm_store_data(const BdrvDirtyBitmap *bitmap, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+hbitmap_store_data(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dbm_restore_data(BdrvDirtyBitmap *bitmap, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+hbitmap_restore_data(bitmap->bitmap, buf, start, count);
+}
+
+BdrvDirtyBitmap **bdrv_dbm_find_all_named(BlockDriverState *bs, int *count)
+{
+BdrvDirtyBitmap *bm, **res, **iter;
+assert(count);
+
+QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
+if (bm->name != NULL) {
+(*count)++;
+}
+}
+
+iter = res = g_malloc(sizeof(*res) * (*count));
+QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
+if (bm->name != NULL) {
+*iter++ = bm;
+}
+}
+
+return res;
+}
+
+void bdrv_dbm_restore_finish(void)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bm;
+
+for (bs = bdrv_next(NULL); bs != NULL; bs = bdrv_next(bs)) {
+QLIST_FOREACH(bm, &bs->dirty_bitmaps, list) {
+if (bm->name != NULL) {
+hbitmap_restore_finish(bm->bitmap);
+}
+}
+}
+}
+
+void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
+int nr_sectors)
{
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
diff --git a/include/block/block.h b/include/block/block.h
index b21233c..09eff80 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -459,6 +459,16 @@ void bdrv_dirty_iter_init(BlockDriverState *bs,
void bdrv_dirty_iter_set(struct HBitmapIter *hbi, int64_t offset);
int64_t bdrv_get_dirty_count(BlockDriverState *bs, BdrvDirtyBitmap *bitmap);
+uint64_t bdrv_dbm_data_size(const BdrvDirtyBitmap *bitmap, uint64_t count);
+void bdrv_dbm_store_data(const BdrvDirtyBitmap *bitmap, uint8_t *buf,
+ uint64_t start, uint64_t count);
+void bdrv_dbm_restore_data(BdrvDirtyBitmap *bitmap, uint8_t *buf,
+ uint64_t start, uint64_t count);
+bool bdrv_dbm_is_named(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+BdrvDirtyBitmap **bdrv_dbm_find_all_named(BlockDriverState *bs, int *count);
+void bdrv_dbm_restore_finish(void);
+
void bdrv_enable_copy_on_read(BlockDriverState *bs);
void bdrv_disable_copy_on_read(BlockDriverState *bs);
Re: [Qemu-devel] [PATCH 6/9] block-migration: tiny refactoring
On 09.01.2015 00:23, John Snow wrote:
On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote:
Add blk_create and blk_free to remove code duplicates. Otherwise,
duplicates will rise in the following patches because of BlkMigBlock
sturcture extendin.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block-migration.c | 56
+--
1 file changed, 30 insertions(+), 26 deletions(-)
diff --git a/block-migration.c b/block-migration.c
index 5b4aa0f..d0c825f 100644
--- a/block-migration.c
+++ b/block-migration.c
@@ -113,6 +113,30 @@ static void blk_mig_unlock(void)
qemu_mutex_unlock(&block_mig_state.lock);
}
+/* Only allocating and initializing structure fields, not copying
any data. */
+
+static BlkMigBlock *blk_create(BlkMigDevState *bmds, int64_t sector,
+int nr_sectors)
+{
+BlkMigBlock *blk = g_new(BlkMigBlock, 1);
+blk->buf = g_malloc(BLOCK_SIZE);
+blk->bmds = bmds;
+blk->sector = sector;
+blk->nr_sectors = nr_sectors;
+
+blk->iov.iov_base = blk->buf;
+blk->iov.iov_len = nr_sectors * BDRV_SECTOR_SIZE;
+qemu_iovec_init_external(&blk->qiov, &blk->iov, 1);
+
+return blk;
+}
+
+static void blk_free(BlkMigBlock *blk)
+{
+g_free(blk->buf);
+g_free(blk);
+}
+
/* Must run outside of the iothread lock during the bulk phase,
* or the VM will stall.
*/
@@ -285,15 +309,7 @@ static int mig_save_device_bulk(QEMUFile *f,
BlkMigDevState *bmds)
nr_sectors = total_sectors - cur_sector;
}
-blk = g_new(BlkMigBlock, 1);
-blk->buf = g_malloc(BLOCK_SIZE);
-blk->bmds = bmds;
-blk->sector = cur_sector;
-blk->nr_sectors = nr_sectors;
-
-blk->iov.iov_base = blk->buf;
-blk->iov.iov_len = nr_sectors * BDRV_SECTOR_SIZE;
-qemu_iovec_init_external(&blk->qiov, &blk->iov, 1);
+blk = blk_create(bmds, cur_sector, nr_sectors);
blk_mig_lock();
block_mig_state.submitted++;
@@ -467,17 +483,9 @@ static int mig_save_device_dirty(QEMUFile *f,
BlkMigDevState *bmds,
} else {
nr_sectors = BDRV_SECTORS_PER_DIRTY_CHUNK;
}
-blk = g_new(BlkMigBlock, 1);
-blk->buf = g_malloc(BLOCK_SIZE);
-blk->bmds = bmds;
-blk->sector = sector;
-blk->nr_sectors = nr_sectors;
+blk = blk_create(bmds, sector, nr_sectors);
if (is_async) {
-blk->iov.iov_base = blk->buf;
-blk->iov.iov_len = nr_sectors * BDRV_SECTOR_SIZE;
-qemu_iovec_init_external(&blk->qiov, &blk->iov, 1);
-
I suppose in the (!is_async) branch we don't reference iov/qiov again,
but the functional difference caught my eye. It used to only be called
under the "is_async" branch, but now is going to be executed
unconditionally.
Is that fine?
It think it doesn't matter. I can add a parameter 'is_async' to
blk_create(), but what is worse - excess parameter or excess
initialization? And why not to initialize the whole structure in
blk_create() unconditionally?
Re: [Qemu-devel] [PATCH 7/9] block-migration: remove not needed iothread lock
On 09.01.2015 00:24, John Snow wrote: On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote: Instead of locking iothread, we can just swap these calls. So, if some write to our range occures before resetting the bitmap, then it will get into subsequent aio read, becouse it occures, in any case, after resetting the bitmap. s/occures/occurs/g s/becouse/because/g (I hope you are not annoyed by the spelling corrections: They are in good faith and personally I would hope people would correct any of my spelling mistakes before it goes in the git log!) Signed-off-by: Vladimir Sementsov-Ogievskiy --- block-migration.c | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/block-migration.c b/block-migration.c index d0c825f..908a66d 100644 --- a/block-migration.c +++ b/block-migration.c @@ -315,13 +315,11 @@ static int mig_save_device_bulk(QEMUFile *f, BlkMigDevState *bmds) block_mig_state.submitted++; blk_mig_unlock(); -qemu_mutex_lock_iothread(); +bdrv_reset_dirty_bitmap(bs, bmds->dirty_bitmap, cur_sector, nr_sectors); + blk->aiocb = bdrv_aio_readv(bs, cur_sector, &blk->qiov, nr_sectors, blk_mig_read_cb, blk); -bdrv_reset_dirty_bitmap(bs, bmds->dirty_bitmap, cur_sector, nr_sectors); -qemu_mutex_unlock_iothread(); - bmds->cur_sector = cur_sector + nr_sectors; return (bmds->cur_sector >= total_sectors); } OK, so the justification here is that by ordering it as "reset, read" that any writes that occur after the reset will be simply included in the read, and we won't have reset any bits that we didn't actually get a chance to read. OK. But what about losing the ability to clear bits that are set needlessly? e.g.: Sector 1 is dirty. Sector 2 is clean. We reset the bitmap. All sectors are clean. A write occurs to sector 2, marking it dirty. We read sectors one and two. Sector two is now erroneously marked dirty, when it isn't. It's not a data integrity problem, but we're swapping one inefficiency for another. Do you have a justification for why this is better than the lock? I think without lock is always better. Lock-less operations are usually better, aren't they? So, I didn't think about such kind of efficiency, but only about that this lock is needless. If we don't swap these reset and read, lock is necessary of course. Furthermore, If we have a lock, this write (to sector 2) will happen after unlock, and the sector will be dirty anyway. So, this efficiency is the same, but we can remove lock.
Re: [Qemu-devel] [PATCH 7/9] block-migration: remove not needed iothread lock
Best regards, Vladimir On 09.01.2015 01:28, Paolo Bonzini wrote: On 11/12/2014 15:17, Vladimir Sementsov-Ogievskiy wrote: -qemu_mutex_lock_iothread(); +bdrv_reset_dirty_bitmap(bs, bmds->dirty_bitmap, cur_sector, nr_sectors); + blk->aiocb = bdrv_aio_readv(bs, cur_sector, &blk->qiov, nr_sectors, blk_mig_read_cb, blk); -bdrv_reset_dirty_bitmap(bs, bmds->dirty_bitmap, cur_sector, nr_sectors); -qemu_mutex_unlock_iothread(); - bdrv_aio_readv is not thread safe, so it needs the iothread lock. bdrv_reset_dirty_bitmap is also not thread safe, because bmds->dirty_bitmap is accessed concurrently by bdrv_reset_dirty (discard) and bdrv_set_dirty (write). So it needs the iothread lock too. Moving bdrv_reset_dirty_bitmap before the read is a good idea. Paolo Ok.
Re: [Qemu-devel] [PATCH 9/9] block-migration: add named dirty bitmaps migration
Best regards,
Vladimir
On 09.01.2015 01:05, John Snow wrote:
CCing migration maintainers, feedback otherwise in-line.
On 12/11/2014 09:17 AM, Vladimir Sementsov-Ogievskiy wrote:
Just migrate parts of dirty bitmaps, corresponding to the block being
migrated. Also, skip block migration if it is disabled (blk parameter
of migrate command is false).
Skipping shared sectors: bitmaps are migrated independently of this,
just send blk without block data.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
In terms of general approach, migrating the dirty bitmap alongside the
blocks it describes makes sense when migrating both.
Is this a lot of overhead when that's not the case, though? If we
utilize the "bitmap only" pathways added here, don't we iterate
through the savevm handlers a lot to only transfer very little data
per section?
If we really need migration of bitmaps apart from the data they
describe, is it not worth developing a more condensed transfer
mechanism to get more bitmap data per iteration, instead of just one
block's worth?
The first stage of migration can be easily optimized in this case - just
take a larger bitmap pieces. For the second stage, it's not as simple.
If we will take larger pieces on each step - we will just increase dirty
data transfer. One approach is to maintain "dirty-region" - two numbers,
representing the region of set bits in migration dirty bitmap, and send
data only when this region is large enough.
block-migration.c | 173
+++---
savevm.c | 1 +
2 files changed, 154 insertions(+), 20 deletions(-)
diff --git a/block-migration.c b/block-migration.c
index 908a66d..95d54a1 100644
--- a/block-migration.c
+++ b/block-migration.c
@@ -32,6 +32,8 @@
#define BLK_MIG_FLAG_EOS0x02
#define BLK_MIG_FLAG_PROGRESS 0x04
#define BLK_MIG_FLAG_ZERO_BLOCK 0x08
+#define BLK_MIG_FLAG_HAS_BLOCK 0x10
+#define BLK_MIG_FLAG_HAS_BITMAPS0x20
OK: As a result of allowing bitmaps to be migrated without the block
data itself, we now must acknowledge the concept that we can send
either block data, bitmaps, both, or neither -- so new defines are
added here to indicate what data can be found in the section following.
#define MAX_IS_ALLOCATED_SEARCH 65536
@@ -51,6 +53,8 @@ typedef struct BlkMigDevState {
int shared_base;
int64_t total_sectors;
QSIMPLEQ_ENTRY(BlkMigDevState) entry;
+int nr_bitmaps;
+BdrvDirtyBitmap **dirty_bitmaps;
/* Only used by migration thread. Does not need a lock. */
int bulk_completed;
@@ -64,6 +68,11 @@ typedef struct BlkMigDevState {
Error *blocker;
} BlkMigDevState;
+typedef struct BlkMigDirtyBitmap {
+BdrvDirtyBitmap *bitmap;
+uint8_t *buf;
+} BlkMigDirtyBitmap;
+
typedef struct BlkMigBlock {
/* Only used by migration thread. */
uint8_t *buf;
@@ -74,6 +83,9 @@ typedef struct BlkMigBlock {
QEMUIOVector qiov;
BlockAIOCB *aiocb;
+int nr_bitmaps;
+BlkMigDirtyBitmap *dirty_bitmaps;
+
/* Protected by block migration lock. */
int ret;
QSIMPLEQ_ENTRY(BlkMigBlock) entry;
@@ -83,6 +95,7 @@ typedef struct BlkMigState {
/* Written during setup phase. Can be read without a lock. */
int blk_enable;
int shared_base;
+int dbm_enable;
Similar to the feedback in a previous patch, we may not want to use
'dbm' to mean "Dirty Bitmap" because we have not been applying the
abbreviation consistently.
For now, the recommendation from stefan is to use the full
"bdrv_dirty_bitmap" or "dirty_bitmap" in function names.
If we do want an acronym to refer to this particular type of dirty
bitmap, we should apply it consistently to all functions that work
with the BdrvDirtyBitmap type.
For now, "bdrv_dirty_bitmap_enable" should suffice, even though it's a
bit wordy.
It's a flag, that enables the migration of dirty bitmaps, like
blk_enable enables the migrtion of blocks.. Then, should it be
"dirty_bitmap_migration_enable"? Isn't it too long for a simple flag
variable?
QSIMPLEQ_HEAD(bmds_list, BlkMigDevState) bmds_list;
int64_t total_sector_sum;
bool zero_blocks;
@@ -116,27 +129,63 @@ static void blk_mig_unlock(void)
/* Only allocating and initializing structure fields, not copying
any data. */
static BlkMigBlock *blk_create(BlkMigDevState *bmds, int64_t sector,
-int nr_sectors)
+ int nr_sectors, bool only_bitmaps)
{
+int i;
BlkMigBlock *blk = g_new(BlkMigBlock, 1);
-blk->buf = g_malloc(BLOCK_SIZE);
+blk->buf = only_bitmaps ? NULL : g_malloc(BLOCK_SIZE);
blk->bmds = bmds;
blk->sector = sector;
blk->nr_sectors = nr_sectors;
+blk->dirty_
Re: [Qemu-devel] [PATCH v6 02/10] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
+if (!name || name[0] == '\0') {
Isn't is better to move "name[0] == '\0'" check to
bdrv_create_dirty_bitmap, near existed name checking?
+if (granularity < 512 || is_power_of_2(granularity)) {
+error_setg(errp, "Granularity must be power of 2 "
+ "and greater than 512");
+return;
+}
+} else {
+granularity = 65536;
+}
Why not using something like DEFAULT_CLUSTER_SIZE, as in block/qcow2.h ?
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH v6 05/10] block: Add bdrv_copy_dirty_bitmap and bdrv_reset_dirty_bitmap
from [PATCH v6 02/10]
+void qmp_block_dirty_bitmap_remove(const char *device, const char *name,
+ Error **errp)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+
+bs = bdrv_find(device);
+if (!bs) {
+error_set(errp, QERR_DEVICE_NOT_FOUND, device);
+return;
+}
+
+if (!name || name[0] == '\0') {
+error_setg(errp, "Bitmap name cannot be empty");
+return;
+}
+bitmap = bdrv_find_dirty_bitmap(bs, name);
+if (!bitmap) {
+error_setg(errp, "Dirty bitmap not found: %s", name);
+return;
+}
+
+bdrv_dirty_bitmap_make_anon(bs, bitmap);
+bdrv_release_dirty_bitmap(bs, bitmap);
+}
from [PATCH v6 05/10]:
+void qmp_block_dirty_bitmap_enable(const char *device, const char *name,
+ Error **errp)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+
+bs = bdrv_find(device);
+if (!bs) {
+error_set(errp, QERR_DEVICE_NOT_FOUND, device);
+return;
+}
+
+bitmap = bdrv_find_dirty_bitmap(bs, name);
+if (!bitmap) {
+error_setg(errp, "Dirty bitmap not found: %s", name);
+return;
+}
+
+bdrv_enable_dirty_bitmap(bs, bitmap);
+}
+
+void qmp_block_dirty_bitmap_disable(const char *device, const char *name,
+Error **errp)
+{
+BlockDriverState *bs;
+BdrvDirtyBitmap *bitmap;
+
+bs = bdrv_find(device);
+if (!bs) {
+error_set(errp, QERR_DEVICE_NOT_FOUND, device);
+return;
+}
+
+bitmap = bdrv_find_dirty_bitmap(bs, name);
+if (!bitmap) {
+error_setg(errp, "Dirty bitmap not found: %s", name);
+return;
+}
+
+bdrv_disable_dirty_bitmap(bs, bitmap);
+}
+
there is one inconsistence:
you have check
+if (!name || name[0] == '\0') {
+error_setg(errp, "Bitmap name cannot be empty");
+return;
+}
when accessing bitmap in qmp_block_dirty_bitmap_remove, but not in
qmp_block_dirty_bitmap_{enable,disable}.
Also, I think it'll be better to put similar part of these three
functions into one separate function to avoid duplicates, like
static BdrvDirtyBitmap *bitmap find_dirty_bitmap(const char *device, const char
*name,
Error **errp)
{
BlockDriverState *bs;
BdrvDirtyBitmap *bitmap;
// most simple error condition earlier
if (!name || name[0] == '\0') {
error_setg(errp, "Bitmap name cannot be empty");
return NULL;
}
bs = bdrv_find(device);
if (!bs) {
error_set(errp, QERR_DEVICE_NOT_FOUND, device);
return NULL;
}
bitmap = bdrv_find_dirty_bitmap(bs, name);
if (!bitmap) {
error_setg(errp, "Dirty bitmap not found: %s", name);
return NULL;
}
return bitmap;
}
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH v6 07/10] qmp: Add support of "dirty-bitmap" sync mode for drive-backup
@@ -317,7 +321,21 @@ static void coroutine_fn backup_run(void *opaque)
if (alloced == 0) {
continue;
}
+} else if (job->sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) {
+int i, dirty = 0;
+for (i = 0; i < BACKUP_SECTORS_PER_CLUSTER;
+ i += job->sync_bitmap_gran) {
+if (bdrv_get_dirty(bs, job->sync_bitmap,
+start * BACKUP_SECTORS_PER_CLUSTER + i)) {
+dirty = 1;
+break;
+}
+}
+if (!dirty) {
+continue;
+}
}
+
With such solution we don't use the power of HBitmap, which provides
"hbitmap_iter_skip_words", with O(log(n)) complexity. Here we look
through the whole bitmap (O(n)), instead of jumping to the next dirty bit.
--
С уважением,
Владимир
Re: [Qemu-devel] [PATCH v6 04/10] hbitmap: Add hbitmap_copy
+
+HBitmap *hbitmap_copy(const HBitmap *bitmap)
+{
+int i;
+int64_t size;
+HBitmap *hb = g_memdup(bitmap, sizeof(struct HBitmap));
+
+size = bitmap->size;
+for (i = HBITMAP_LEVELS; i-- > 0; ) {
+size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+hb->levels[i] = g_memdup(bitmap->levels[i],
+ size * sizeof(unsigned long));
+}
+
+return hb;
+}
"(size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL" - will be zero iff size == 0. Is
it really possible in qemu? If not, we doesn't need MAX(..., 1).
There is similar construction in older "hbitmap_alloc" function.
--
Best regards,
Vladimir
Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series
(2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? - The name of the bitmap and also the size of this name (5) Partial Persistence We did not discuss only saving higher levels of the bitmap. What's the primary benefit you're seeking? Hmm. It may be used for faster sync. Maybe, save some of bitmap levels on timer while vm is running and save the last level on shutdown? CC qemu-devel - ok. Best regards, Vladimir On 18.11.2014 02:46, John Snow wrote: On 11/13/2014 08:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hi I'd just like to start working on persistent dirty bitmap. My thoughts about it are the following: - qemu -drive file=file,dirty_bitmap=file so, bitmap will be loaded with drive open and saved with drive close. - save only meaningful (the last) level of the bitmap, restore all levels on bitmap loading - bool parameter "persistent" for bdrv_create_dirty_bitmap and BdrvDirtyBitmap - internal dirty_bitmaps, saved in qcow2 file Best regards, Vladimir I am thinking: (1) Command Lines If you enable dirty bitmaps and give it a file that doesn't exist, it should error out on you. If you enable dirty bitmaps and give it a file that's blank, it understands that it is to create a persistent bitmap file in this location and it should enable persistence. If a bitmap file is given and it has valid magic, this should imply persistence. I am hesitant to have it auto-create files that don't already exist in case the files become large in size and a misconfiguration leads to repeated creation of these files that get orphaned in random folders. Perhaps we can add a create=auto flag or similar to allow this behavior if wanted. (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? (3) Data Integrity The dirty flag could work something like: - If, on first open, the file has the dirty flag set, we need to discard the bitmap data because we can no longer trust it. - If the bitmap file is clean, proceed as normal, but take a lock against any of the bitmap functions to prevent them from marking any bits dirty. - On first write to a clean persistent bitmap, delay the write until we can mark the bitmap as dirty first. This incurs a write penalty when we try to use the bitmap at first... - Unlock the bitmap functions and allow them to mark blocks as needed. - At some point, based on a sync policy, re-commit the dirty information to the file and mark the file as clean once more and re-take the persistence lock. (4) Synchronization Policy - Sync after so many bits become dirty in the bitmap, either as an absolute threshold or a density percentage? - Sync periodically on a fixed timer? - Sync periodically opportunistically when I/O utilization becomes relatively low? (With some sort of starvation prevention timer?) - Sync only at shutdown? In discussing with Stefan, I think we rather liked the idea of a timer that tries to re-commit the block data during lulls in the I/O. (5) Partial Persistence We did not discuss only saving higher levels of the bitmap. What's the primary benefit you're seeking? (6) Inclusion as qcow2 Metadata And lastly, we did discuss the inclusion of the bitmap as qcow2 metadata, but decided it wasn't our principle target for the format to allow generality to other file formats. We didn't really discuss the idea of having it as an option or an extension, but I don't (off the top of my head) have any reasonings against it, but I will likely not work on it myself. You didn't CC qemu-devel on this (so I won't!), but perhaps we should re-send out our ideas to the wider list for feedback before we proceed any further. Maybe we can split the work if we agree upon a design. Thanks! --js P.S.: I'm still cleaning up Fam's first patchset based on Max's and your feedback. Hope to have it out by the end of this week. On 11.11.2014 18:59, John Snow wrote: On 11/10/2014 03:15 AM, Vladimir Sementsov-Ogievskiy wrote: Hi Fam, hi Jorn. Jagane's project - http://wiki.qemu.org/Features/Livebackup In two words: Normal delta - like in qemu, while backuping, we save all new writes to separate virtual disk - delta. When backup is done, we can merge delta back to original image. Reverse delta - w
Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series
(3) Data Integrity The dirty flag could work something like: - If, on first open, the file has the dirty flag set, we need to discard the bitmap data because we can no longer trust it. - If the bitmap file is clean, proceed as normal, but take a lock against any of the bitmap functions to prevent them from marking any bits dirty. - On first write to a clean persistent bitmap, delay the write until we can mark the bitmap as dirty first. This incurs a write penalty when we try to use the bitmap at first... - Unlock the bitmap functions and allow them to mark blocks as needed. - At some point, based on a sync policy, re-commit the dirty information to the file and mark the file as clean once more and re-take the persistence lock. Correct me if I'm wrong. #Read bitmap: read in blockdev_init, before any write to device, so no lock is needed. #Set bits in bitmap: if bitmap.dirty_flag: set bits else: LOCK set bits set bitmap.dirty_flag set dirty_flag in bitmap file UNLOCK #Sync: if not bitmap.dirty_flag: skip sync else: LOCK save one of bitmap levels (saving the last one is too long and not very good idea, because it is fast-updateing) unset dirty_flag in bitmap file unset bitmap.dirty_flag UNLOCK #Last sync in bdrv_close: Just save the last bitmap level and unset dirty_flag in bitmap file Also.. I'm not quite sure about locking.. As I understand, co-routines in qemu are not running in parallel, is locking required? Or sync timer will not be co-routine based? Best regards, Vladimir On 18.11.2014 13:54, Vladimir Sementsov-Ogievskiy wrote: (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? - The name of the bitmap and also the size of this name (5) Partial Persistence We did not discuss only saving higher levels of the bitmap. What's the primary benefit you're seeking? Hmm. It may be used for faster sync. Maybe, save some of bitmap levels on timer while vm is running and save the last level on shutdown? CC qemu-devel - ok. Best regards, Vladimir On 18.11.2014 02:46, John Snow wrote: On 11/13/2014 08:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hi I'd just like to start working on persistent dirty bitmap. My thoughts about it are the following: - qemu -drive file=file,dirty_bitmap=file so, bitmap will be loaded with drive open and saved with drive close. - save only meaningful (the last) level of the bitmap, restore all levels on bitmap loading - bool parameter "persistent" for bdrv_create_dirty_bitmap and BdrvDirtyBitmap - internal dirty_bitmaps, saved in qcow2 file Best regards, Vladimir I am thinking: (1) Command Lines If you enable dirty bitmaps and give it a file that doesn't exist, it should error out on you. If you enable dirty bitmaps and give it a file that's blank, it understands that it is to create a persistent bitmap file in this location and it should enable persistence. If a bitmap file is given and it has valid magic, this should imply persistence. I am hesitant to have it auto-create files that don't already exist in case the files become large in size and a misconfiguration leads to repeated creation of these files that get orphaned in random folders. Perhaps we can add a create=auto flag or similar to allow this behavior if wanted. (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? (3) Data Integrity The dirty flag could work something like: - If, on first open, the file has the dirty flag set, we need to discard the bitmap data because we can no longer trust it. - If the bitmap file is clean, proceed as normal, but take a lock against any of the bitmap functions to prevent them from marking any bits dirty. - On first write to a clean persistent bitmap, delay the write until we can mark the bitmap as dirty first. This incurs a write penalty when we try to use the bitmap at first... - Unlock the bitmap functions and allow them to mark blocks as needed. - At some point, based on a sync policy, re-commit the dirty information to the file and mark the file as clean once more and re-take the persistence lock. (4) Synchronization Policy - Sync after so many bits become dirty in the bitmap, either as an absolute threshold or a density percentage? - Sync periodically on a fixed tim
Re: [Qemu-devel] [PATCH v6 00/10] block: Incremental backup series
Also, if we sync not the last level, bitmap.dirty_flag should be related to syncing level, not to the whole bitmap, to reduce sync overhead. Or, if we implement difficult sync policy, there should be dirty flags for each bitmap level. Despite this, only one level is needed to be saved in the bitmap file. PS: more ideas about file format - thanks to Denis V. Lunev 1) Shouldn't we consider a possibility of storing several bitmaps in one file? Or one bitmap = one file? 2) Implement header extensions like in qcow2. Best regards, Vladimir On 18.11.2014 16:09, Vladimir Sementsov-Ogievskiy wrote: (3) Data Integrity The dirty flag could work something like: - If, on first open, the file has the dirty flag set, we need to discard the bitmap data because we can no longer trust it. - If the bitmap file is clean, proceed as normal, but take a lock against any of the bitmap functions to prevent them from marking any bits dirty. - On first write to a clean persistent bitmap, delay the write until we can mark the bitmap as dirty first. This incurs a write penalty when we try to use the bitmap at first... - Unlock the bitmap functions and allow them to mark blocks as needed. - At some point, based on a sync policy, re-commit the dirty information to the file and mark the file as clean once more and re-take the persistence lock. Correct me if I'm wrong. #Read bitmap: read in blockdev_init, before any write to device, so no lock is needed. #Set bits in bitmap: if bitmap.dirty_flag: set bits else: LOCK set bits set bitmap.dirty_flag set dirty_flag in bitmap file UNLOCK #Sync: if not bitmap.dirty_flag: skip sync else: LOCK save one of bitmap levels (saving the last one is too long and not very good idea, because it is fast-updateing) unset dirty_flag in bitmap file unset bitmap.dirty_flag UNLOCK #Last sync in bdrv_close: Just save the last bitmap level and unset dirty_flag in bitmap file Also.. I'm not quite sure about locking.. As I understand, co-routines in qemu are not running in parallel, is locking required? Or sync timer will not be co-routine based? Best regards, Vladimir On 18.11.2014 13:54, Vladimir Sementsov-Ogievskiy wrote: (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? - The name of the bitmap and also the size of this name (5) Partial Persistence We did not discuss only saving higher levels of the bitmap. What's the primary benefit you're seeking? Hmm. It may be used for faster sync. Maybe, save some of bitmap levels on timer while vm is running and save the last level on shutdown? CC qemu-devel - ok. Best regards, Vladimir On 18.11.2014 02:46, John Snow wrote: On 11/13/2014 08:54 AM, Vladimir Sementsov-Ogievskiy wrote: Hi I'd just like to start working on persistent dirty bitmap. My thoughts about it are the following: - qemu -drive file=file,dirty_bitmap=file so, bitmap will be loaded with drive open and saved with drive close. - save only meaningful (the last) level of the bitmap, restore all levels on bitmap loading - bool parameter "persistent" for bdrv_create_dirty_bitmap and BdrvDirtyBitmap - internal dirty_bitmaps, saved in qcow2 file Best regards, Vladimir I am thinking: (1) Command Lines If you enable dirty bitmaps and give it a file that doesn't exist, it should error out on you. If you enable dirty bitmaps and give it a file that's blank, it understands that it is to create a persistent bitmap file in this location and it should enable persistence. If a bitmap file is given and it has valid magic, this should imply persistence. I am hesitant to have it auto-create files that don't already exist in case the files become large in size and a misconfiguration leads to repeated creation of these files that get orphaned in random folders. Perhaps we can add a create=auto flag or similar to allow this behavior if wanted. (2) File Format Some standard file magic, which includes: - Some magic byte(s) - Dirty flag. Needed to tell if we can trust this data or not. - The size of the bitmap - The granularity of the bitmap - The offset to the first sector of bitmap data (Maybe? It can't hurt if we give ourselves a sector's worth to write metadata within.) - Data starting at... PAGESIZE? (3) Data Integrity The dirty flag could work something like: - If, on first open, the file has the dirty flag set, we need to discard the bitmap data because we can no longer trust it. - If the bitmap file is clean, proceed as normal, but take a lock against any of the bitmap functions to prevent them from markin
[Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
QDB file is for storing dirty bitmap. The specification is based on
qcow2 specification.
Saving several bitmaps is necessary when server shutdowns during
backup. In this case 2 tables for each disk are available. One
collected for a previous period and one active. Though this feature
is discussable.
Big endian format and Standard Cluster Descriptor are used to simplify
integration with qcow2, to support internal bitmaps for qcow2 in future.
The idea is that the same procedure writing the data to QDB file could
do the same for QCOW2. The only difference is cluster refcount table.
Should we use it here or not is still questionable.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
docs/specs/qdb.txt | 132 +
1 file changed, 132 insertions(+)
create mode 100644 docs/specs/qdb.txt
diff --git a/docs/specs/qdb.txt b/docs/specs/qdb.txt
new file mode 100644
index 000..d570a69
--- /dev/null
+++ b/docs/specs/qdb.txt
@@ -0,0 +1,132 @@
+== General ==
+
+"QDB" means "Qemu Dirty Bitmaps". QDB file can store several dirty bitmaps.
+QDB file is organized in units of constant size, which are called clusters.
+
+All numbers in QDB are stored in Big Endian byte order.
+
+== Header ==
+
+The first cluster of a QDB image contains the file header:
+
+Byte 0 - 3: magic
+QDB magic string ("QDB\0")
+
+ 4 - 7: version
+Version number (valid value is 1)
+
+ 8 - 11: cluster_bits
+Number of bits that are used for addressing an offset
+within a cluster (1 << cluster_bits is the cluster size).
+Must not be less than 9 (i.e. 512 byte clusters).
+
+ 12 - 15: nb_bitmaps
+Number of bitmaps contained in the file
+
+ 16 - 23: bitmaps_offset
+Offset into the QDB file at which the bitmap table starts.
+Must be aligned to a cluster boundary.
+
+ 24 - 27: header_length
+Length of the header structure in bytes.
+
+Like in qcow2, directly after the image header, optional sections called
header extensions can
+be stored. Each extension has a structure like the following:
+
+Byte 0 - 3: Header extension type:
+0x - End of the header extension area
+other - Unknown header extension, can be safely
+ ignored
+
+ 4 - 7: Length of the header extension data
+
+ 8 - n: Header extension data
+
+ n - m: Padding to round up the header extension size to the next
+multiple of 8.
+
+Unless stated otherwise, each header extension type shall appear at most once
+in the same image.
+
+== Cluster mapping ==
+
+QDB uses a ONE-level structure for the mapping of
+bitmaps to host clusters. It is called L1 table.
+
+The L1 table has a variable size (stored in the Bitmap table entry) and may
+use multiple clusters, however it must be contiguous in the QDB file.
+
+Given a offset into the bitmap, the offset into the QDB file can be
+obtained as follows:
+
+offset = l1_table[offset / cluster_size] + (offset % cluster_size)
+
+L1 table entry:
+
+Bit 0 - 61: Cluster descriptor
+
+62 - 63: Reserved
+
+Standard Cluster Descriptor (the same as in qcow2):
+
+Bit 0:If set to 1, the cluster reads as all zeros. The host
+cluster offset can be used to describe a preallocation,
+but it won't be used for reading data from this cluster,
+nor is data read from the backing file if the cluster is
+unallocated.
+
+ 1 - 8:Reserved (set to 0)
+
+ 9 - 55:Bits 9-55 of host cluster offset. Must be aligned to a
+cluster boundary. If the offset is 0, the cluster is
+unallocated.
+
+56 - 61:Reserved (set to 0)
+
+If a cluster is unallocated, read requests shall read zero.
+
+== Bitmap table ==
+
+QDB supports storing of several bitmaps.
+
+A directory of all bitmaps is stored in the bitmap table, a contiguous area
+in the QDB file, whose starting offset and length are given by the header
+fields bitmaps_offset and nb_bitmaps. The entries of the bitmap table
+have variable length, depending on the length of name and extra data.
+
+Bitmap table entry:
+
+Byte 0 - 7:Offset into the QDB file at which the L1 table for the
+bitmap starts. Must be aligned to a cluster boundary.
+
+ 8 - 11:Number of entries in the L1 table of the bitmap
+
+12 - 15:Bitmap granularity
+As represented in HBitmap structure. Given a granularity of
+G, each bit in the bitmap will actually represent a group
+of 2^G
Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
Also, it may be better to make this as qcow2 extension. And bitmap will
be saved in separate qcow2 file, which will contain only the bitmap(s)
and no other data (no disk, no snapshots).
Best regards,
Vladimir
On 20.11.2014 13:34, Vladimir Sementsov-Ogievskiy wrote:
QDB file is for storing dirty bitmap. The specification is based on
qcow2 specification.
Saving several bitmaps is necessary when server shutdowns during
backup. In this case 2 tables for each disk are available. One
collected for a previous period and one active. Though this feature
is discussable.
Big endian format and Standard Cluster Descriptor are used to simplify
integration with qcow2, to support internal bitmaps for qcow2 in future.
The idea is that the same procedure writing the data to QDB file could
do the same for QCOW2. The only difference is cluster refcount table.
Should we use it here or not is still questionable.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
docs/specs/qdb.txt | 132 +
1 file changed, 132 insertions(+)
create mode 100644 docs/specs/qdb.txt
diff --git a/docs/specs/qdb.txt b/docs/specs/qdb.txt
new file mode 100644
index 000..d570a69
--- /dev/null
+++ b/docs/specs/qdb.txt
@@ -0,0 +1,132 @@
+== General ==
+
+"QDB" means "Qemu Dirty Bitmaps". QDB file can store several dirty bitmaps.
+QDB file is organized in units of constant size, which are called clusters.
+
+All numbers in QDB are stored in Big Endian byte order.
+
+== Header ==
+
+The first cluster of a QDB image contains the file header:
+
+Byte 0 - 3: magic
+QDB magic string ("QDB\0")
+
+ 4 - 7: version
+Version number (valid value is 1)
+
+ 8 - 11: cluster_bits
+Number of bits that are used for addressing an offset
+within a cluster (1 << cluster_bits is the cluster size).
+Must not be less than 9 (i.e. 512 byte clusters).
+
+ 12 - 15: nb_bitmaps
+Number of bitmaps contained in the file
+
+ 16 - 23: bitmaps_offset
+Offset into the QDB file at which the bitmap table starts.
+Must be aligned to a cluster boundary.
+
+ 24 - 27: header_length
+Length of the header structure in bytes.
+
+Like in qcow2, directly after the image header, optional sections called
header extensions can
+be stored. Each extension has a structure like the following:
+
+Byte 0 - 3: Header extension type:
+0x - End of the header extension area
+other - Unknown header extension, can be safely
+ ignored
+
+ 4 - 7: Length of the header extension data
+
+ 8 - n: Header extension data
+
+ n - m: Padding to round up the header extension size to the next
+multiple of 8.
+
+Unless stated otherwise, each header extension type shall appear at most once
+in the same image.
+
+== Cluster mapping ==
+
+QDB uses a ONE-level structure for the mapping of
+bitmaps to host clusters. It is called L1 table.
+
+The L1 table has a variable size (stored in the Bitmap table entry) and may
+use multiple clusters, however it must be contiguous in the QDB file.
+
+Given a offset into the bitmap, the offset into the QDB file can be
+obtained as follows:
+
+offset = l1_table[offset / cluster_size] + (offset % cluster_size)
+
+L1 table entry:
+
+Bit 0 - 61: Cluster descriptor
+
+62 - 63: Reserved
+
+Standard Cluster Descriptor (the same as in qcow2):
+
+Bit 0:If set to 1, the cluster reads as all zeros. The host
+cluster offset can be used to describe a preallocation,
+but it won't be used for reading data from this cluster,
+nor is data read from the backing file if the cluster is
+unallocated.
+
+ 1 - 8:Reserved (set to 0)
+
+ 9 - 55:Bits 9-55 of host cluster offset. Must be aligned to a
+cluster boundary. If the offset is 0, the cluster is
+unallocated.
+
+56 - 61:Reserved (set to 0)
+
+If a cluster is unallocated, read requests shall read zero.
+
+== Bitmap table ==
+
+QDB supports storing of several bitmaps.
+
+A directory of all bitmaps is stored in the bitmap table, a contiguous area
+in the QDB file, whose starting offset and length are given by the header
+fields bitmaps_offset and nb_bitmaps. The entries of the bitmap table
+have variable length, depending on the length of name and extra data.
+
+Bitmap table entry:
+
+Byte 0 - 7:Offset into the QDB file at which the L1 table for the
+bitmap starts. Must be aligned to a cluster boundary.
+
+ 8 - 11:Number of entries in
Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
There is a constraint if we want to get live migration for free: The bitmap contents must be accessible with bdrv_read() and bdrv_get_block_status() to skip zero regions. Hm. I'm afraid, it still will not be free. If bitmap is active, it's actual version is in memory. To migrate bitmap file like a disk image, we should start syncing it with every write to corresponding disk, doubling number of io. Moreover, we have normal dirty bitmaps, which have no name/file, do we migrate them? If, for example, the migration occurs when backup in progress? Active bitmaps should be migrated in the same way for persistent/named/normal bitmaps. I can't find in qemu source, is there bitmap migration? Or you are saying about migrating disabled bitmaps? Hm. We should sync bitmap file on bitmap_disable. Disabled persistent bitmap is just a static file ~30mb, we can easily migrate it without common procedure with cow or something like this.. Best regards, Vladimir On 20.11.2014 14:36, Stefan Hajnoczi wrote: On Thu, Nov 20, 2014 at 01:41:14PM +0300, Vladimir Sementsov-Ogievskiy wrote: Also, it may be better to make this as qcow2 extension. And bitmap will be saved in separate qcow2 file, which will contain only the bitmap(s) and no other data (no disk, no snapshots). I think you are on to something with the idea of making the persistent dirty bitmap itself a disk image. That way drive-mirror and other commands can be used to live migrate the dirty bitmap along with the guest's disks. This allows both QEMU and management tools to reuse existing code. (We may need to allow multiple block jobs per BlockDriverState to make this work but in theory that can be done.) There is a constraint if we want to get live migration for free: The bitmap contents must be accessible with bdrv_read() and bdrv_get_block_status() to skip zero regions. Putting the dirty bitmap into its own data structure in qcow2 and not accessible as a BlockDriverState bdrv_read() means custom code must be written to migrate the dirty bitmap. So I suggest putting the bitmap contents into a disk image that can be accessed as a BlockDriverState with bdrv_read(). The metadata (bitmap name, granularity, etc) doesn't need to be stored in the image file because management tools must be aware of it anyway. The only thing besides the data that really needs to be stored is the up-to-date flag to decide whether this dirty bitmap was synced cleanly. A much simpler format would do for that. Stefan
Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
The metadata (bitmap name, granularity, etc) doesn't need to be stored in the image file because management tools must be aware of it anyway. What tools do you mean? In my opinion dirty bitmap should exist as a separate object. If it exists, it should be loaded with it's drive image and it should be maintained by qemu (loaded and enabled as a BdrvDirtyBitmap). If we use qcow2 format for dirty bitmaps, we can store metadata using header extension.. Also snapshots may be used to store several bitmaps in case when server shutdowns during backup and we need to store both current active bitmap and it's snapshot used by backup. Best regards, Vladimir On 20.11.2014 14:36, Stefan Hajnoczi wrote: On Thu, Nov 20, 2014 at 01:41:14PM +0300, Vladimir Sementsov-Ogievskiy wrote: Also, it may be better to make this as qcow2 extension. And bitmap will be saved in separate qcow2 file, which will contain only the bitmap(s) and no other data (no disk, no snapshots). I think you are on to something with the idea of making the persistent dirty bitmap itself a disk image. That way drive-mirror and other commands can be used to live migrate the dirty bitmap along with the guest's disks. This allows both QEMU and management tools to reuse existing code. (We may need to allow multiple block jobs per BlockDriverState to make this work but in theory that can be done.) There is a constraint if we want to get live migration for free: The bitmap contents must be accessible with bdrv_read() and bdrv_get_block_status() to skip zero regions. Putting the dirty bitmap into its own data structure in qcow2 and not accessible as a BlockDriverState bdrv_read() means custom code must be written to migrate the dirty bitmap. So I suggest putting the bitmap contents into a disk image that can be accessed as a BlockDriverState with bdrv_read(). The metadata (bitmap name, granularity, etc) doesn't need to be stored in the image file because management tools must be aware of it anyway. The only thing besides the data that really needs to be stored is the up-to-date flag to decide whether this dirty bitmap was synced cleanly. A much simpler format would do for that. Stefan
Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
Active dirty bitmaps should migrate too. I'm thinking now that the appropriate thing is to add live migration of dirty bitmaps to QEMU (regardless of whether they are active or not). Only for persistent bitmaps, or for all named bitmaps? If for all named bitmaps, then this migration should not be connected with bitmap file and it's format. Best regards, Vladimir On 21.11.2014 19:55, Stefan Hajnoczi wrote: On Fri, Nov 21, 2014 at 01:27:40PM +0300, Vladimir Sementsov-Ogievskiy wrote: There is a constraint if we want to get live migration for free: The bitmap contents must be accessible with bdrv_read() and bdrv_get_block_status() to skip zero regions. Hm. I'm afraid, it still will not be free. If bitmap is active, it's actual version is in memory. To migrate bitmap file like a disk image, we should start syncing it with every write to corresponding disk, doubling number of io. It would be possible to drive-mirror the persistent dirty bitmap and then flush it like all drives when the guest vCPUs are paused for migration. After thinking more about it though, this approach places more I/O into the critical guest downtime phase. In other words, slow disk I/O could lead to long guest downtimes while QEMU tries to write out the dirty bitmap. Moreover, we have normal dirty bitmaps, which have no name/file, do we migrate them? If, for example, the migration occurs when backup in progress? Active bitmaps should be migrated in the same way for persistent/named/normal bitmaps. I can't find in qemu source, is there bitmap migration? bs->dirty_bitmaps is not migrated, in fact none of BlockDriverState is migrated. QEMU only migrates emulated device state (e.g. the hardware registers and associated state). It does not emulate host state that the guest cannot see like the dirty bitmap. Or you are saying about migrating disabled bitmaps? Hm. We should sync bitmap file on bitmap_disable. Disabled persistent bitmap is just a static file ~30mb, we can easily migrate it without common procedure with cow or something like this.. Active dirty bitmaps should migrate too. I'm thinking now that the appropriate thing is to add live migration of dirty bitmaps to QEMU (regardless of whether they are active or not). Stefan
Re: [Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.
I'm thinking now that the
appropriate thing is to add live migration of dirty bitmaps to QEMU
(regardless of whether they are active or not).
Digging the code around, I've found this:
in mig_save_device_dirty which is actually an iteration of live block
migration, after sending a sector we need to clear appropriate bit in
migration dirty bitmap (bmds->dirty_bitmap). But we clear such bits in
all bitmaps, associated with this device:
bdrv_reset_dirty(bmds->bs, sector, nr_sectors);
which is
void bdrv_reset_dirty(BlockDriverState *bs, int64_t cur_sector, int
nr_sectors)
{
BdrvDirtyBitmap *bitmap;
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
hbitmap_reset(bitmap->bitmap, cur_sector, nr_sectors);
}
}
I don't know why is it so, but with such approach we cant talk about
dirty bitmap migration. Actually, all other dirty bitmaps, not related
to this migration are broken because of this.
It's a mistake or I don't understand the concept of several dirty
bitmaps per device in qemu. I've thought that they are separate
entities, which are maintained by qemu. And other subsystems like backup
or migration can create for itself a bitmap and use it not touching
other bitmaps.. Am I wrong?
Best regards,
Vladimir
On 21.11.2014 19:55, Stefan Hajnoczi wrote:
On Fri, Nov 21, 2014 at 01:27:40PM +0300, Vladimir Sementsov-Ogievskiy wrote:
There is a constraint if we want to get live migration for free: The
bitmap contents must be accessible with bdrv_read() and
bdrv_get_block_status() to skip zero regions.
Hm. I'm afraid, it still will not be free. If bitmap is active, it's actual
version is in memory. To migrate bitmap file like a disk image, we should
start syncing it with every write to corresponding disk, doubling number of
io.
It would be possible to drive-mirror the persistent dirty bitmap and
then flush it like all drives when the guest vCPUs are paused for
migration.
After thinking more about it though, this approach places more I/O into
the critical guest downtime phase. In other words, slow disk I/O could
lead to long guest downtimes while QEMU tries to write out the dirty
bitmap.
Moreover, we have normal dirty bitmaps, which have no name/file, do we
migrate them? If, for example, the migration occurs when backup in progress?
Active bitmaps should be migrated in the same way for
persistent/named/normal bitmaps. I can't find in qemu source, is there
bitmap migration?
bs->dirty_bitmaps is not migrated, in fact none of BlockDriverState is
migrated.
QEMU only migrates emulated device state (e.g. the hardware registers
and associated state). It does not emulate host state that the guest
cannot see like the dirty bitmap.
Or you are saying about migrating disabled bitmaps? Hm. We should sync
bitmap file on bitmap_disable. Disabled persistent bitmap is just a static
file ~30mb, we can easily migrate it without common procedure with cow or
something like this..
Active dirty bitmaps should migrate too. I'm thinking now that the
appropriate thing is to add live migration of dirty bitmaps to QEMU
(regardless of whether they are active or not).
Stefan
