On Mon, Jan 13, 2014 at 06:39:39PM +0800, Fam Zheng wrote:
> This implements incremental backup.
>
> A few new QMP commands related to dirty bitmap are added:
>
> dirty-bitmap-add *
> dirty-bitmap-disable *
> dirty-bitmap-remove
>
> (*: also supported as transactions)
>
> As their name implies, they manipulate a block device's dirty bitmap. This
> doesn't interfere with dirty bitmap used for migration, backup, mirror, etc,
> which don't have a name and are invisible to user. Only named bitmaps (created
> by dirty-bitmap-add) can be disabled/removed by user.
>
> They are added to support "user controlled write tracking", so as to determine
> the range of date for incremental backup.
>
> A new sync mode for drive-backup is introduced:
>
> drive-backup device=.. mode=.. sync=dirty-bitmap bitmap=bitmap0
>
> Which will scan dirty bitmap "bitmap0" and only copy all dirty sectors to
> target.
>
> Now, let's see the usage with a simple example:
>
> # Start the guest
> vm = VM()
> vm.start()
>
> # Fake some guest writes with "qemu-io", this is before creating dirty
> # bitmap, so it won't be copied
> vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512k 1M"')
>
> # Create a dirty bitmap to track writes
> vm.qmp("dirty-bitmap-add", device="ide0-hd0", name="dirty-0")
>
> # Fake some more guest writes with "qemu-io", this will be copied
> vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512M 1M"')
>
> # Now "disable" the first dirty bitmap, do the backup according to it,
> # at meantime continue to track dirty with a new dirty bitmap
> vm.qmp("transaction", actions=[
> {
> 'type': 'dirty-bitmap-disable', 'data': {
> 'device': 'ide0-hd0',
> 'name': 'dirty-0'
> }
> }, {
> 'type': 'dirty-bitmap-add', 'data': {
> 'device': 'ide0-hd0',
> 'name': 'dirty-1'
> }
> }, {
> 'type': 'drive-backup', 'data': {
> 'device': 'ide0-hd0',
> 'target': '/tmp/incre.qcow2',
> 'bitmap': 'dirty-0',
> 'sync': 'dirty-bitmap'
> }
> }
> ])
>
> # Once backup job started, the dirty bitmap can be removed (actually only
> # hidden from user since it is still referenced by block job
> vm.qmp("dirty-bitmap-remove", device="ide0-hd0", name="dirty-0")
I'm interested in the lifecycle of a dirty bitmap (but haven't reviewed
the patches yet). In particular, what happens if a bitmap is added to
more than one drive? Is there a more elegant way to handle the disable,
drive-backup, remove step (we only need to explicitly disable because we
still need the bitmap name for the drive-backup command)? Also what
happens if we add the bitmap again after disabling?
No need to answer all these questions, but it suggests the interface
exposes a bit of complexity. Maybe it's possible to make it simpler and
easier to use?
> P.S. Persistent dirty bitmap could be built on top of this series, but is not
> yet implemented, because the storage format and integrity measures are not
> quite clear for now. The discussion is still open and any questions, ideas,
> use
> cases and concerns are all welcome!
It's desirable to keep dirty bitmaps in separate files. That way they
can be used with any image format (raw, qcow2, etc).
For performance, the dirty bitmap is maintained in memory. Keeping the
persistent dirty bitmap consistent would be very expensive since it
requires an fdatasync(2) before each write request.
I think it's reasonable to only write out the in-memory bitmap on
shutdown or live migration. If the QEMU process crashes it is not safe
to trust the dirty bitmap; a full backup must be performed.
The persistent bitmap file must contain:
1. Bitmap granularity (e.g. 64 KB)
2. The actual bitmap (1 TB disk @ 64 KB granularity = 2 MB bitmap)
3. Flags including a "clean" bit
When QEMU activates the persistent dirty bitmap, it clears the "clean"
flag. When QEMU deactivates and finishes writing out the dirty bitmap,
it sets the "clean" flag.
The "clean" flag is used to tell whether the persistent bitmap file is
safe to use again.
The file contains no information about the VM or disk image. It's up to
the user or management tool to keep track of files.
