On Fri, 01/17 17:25, Stefan Hajnoczi wrote: > On Mon, Jan 13, 2014 at 06:39:39PM +0800, Fam Zheng wrote: > > This implements incremental backup. > > > > A few new QMP commands related to dirty bitmap are added: > > > > dirty-bitmap-add * > > dirty-bitmap-disable * > > dirty-bitmap-remove > > > > (*: also supported as transactions) > > > > As their name implies, they manipulate a block device's dirty bitmap. This > > doesn't interfere with dirty bitmap used for migration, backup, mirror, etc, > > which don't have a name and are invisible to user. Only named bitmaps > > (created > > by dirty-bitmap-add) can be disabled/removed by user. > > > > They are added to support "user controlled write tracking", so as to > > determine > > the range of date for incremental backup. > > > > A new sync mode for drive-backup is introduced: > > > > drive-backup device=.. mode=.. sync=dirty-bitmap bitmap=bitmap0 > > > > Which will scan dirty bitmap "bitmap0" and only copy all dirty sectors to > > target. > > > > Now, let's see the usage with a simple example: > > > > # Start the guest > > vm = VM() > > vm.start() > > > > # Fake some guest writes with "qemu-io", this is before creating dirty > > # bitmap, so it won't be copied > > vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512k 1M"') > > > > # Create a dirty bitmap to track writes > > vm.qmp("dirty-bitmap-add", device="ide0-hd0", name="dirty-0") > > > > # Fake some more guest writes with "qemu-io", this will be copied > > vm.hmp('qemu-io ide0-hd0 "write -P 0xa 512M 1M"') > > > > # Now "disable" the first dirty bitmap, do the backup according to it, > > # at meantime continue to track dirty with a new dirty bitmap > > vm.qmp("transaction", actions=[ > > { > > 'type': 'dirty-bitmap-disable', 'data': { > > 'device': 'ide0-hd0', > > 'name': 'dirty-0' > > } > > }, { > > 'type': 'dirty-bitmap-add', 'data': { > > 'device': 'ide0-hd0', > > 'name': 'dirty-1' > > } > > }, { > > 'type': 'drive-backup', 'data': { > > 'device': 'ide0-hd0', > > 'target': '/tmp/incre.qcow2', > > 'bitmap': 'dirty-0', > > 'sync': 'dirty-bitmap' > > } > > } > > ]) > > > > # Once backup job started, the dirty bitmap can be removed (actually > > only > > # hidden from user since it is still referenced by block job > > vm.qmp("dirty-bitmap-remove", device="ide0-hd0", name="dirty-0") > > I'm interested in the lifecycle of a dirty bitmap (but haven't reviewed > the patches yet). In particular, what happens if a bitmap is added to > more than one drive? Is there a more elegant way to handle the disable, > drive-backup, remove step (we only need to explicitly disable because we > still need the bitmap name for the drive-backup command)? Also what > happens if we add the bitmap again after disabling?
A same name on that device can't be used again unless it's removed. A bitmap is associated to (and only) one device, it can't be shared. > > No need to answer all these questions, but it suggests the interface > exposes a bit of complexity. Maybe it's possible to make it simpler and > easier to use? > At least the user has to explicitly start tracking, that's the dirty-bitmap-add step. Alternatively, we can have "disable, drive-backup, remove" step simplified as: drive-backup sync=dirty-bitmap bitmap=dirty0 reset-bitmap=true where backup job copy out the dirty bitmap (and clears it, as reset-bitmap is true), and backup with it atomically. Of course it doesn't have to actually copy the whole bitmap: it just makes the old one anonymous, create a new empty one and give it the same name. When backup is done, the old bitmap is removed. What do you think? Fam > > P.S. Persistent dirty bitmap could be built on top of this series, but is > > not > > yet implemented, because the storage format and integrity measures are not > > quite clear for now. The discussion is still open and any questions, ideas, > > use > > cases and concerns are all welcome! > > It's desirable to keep dirty bitmaps in separate files. That way they > can be used with any image format (raw, qcow2, etc). > > For performance, the dirty bitmap is maintained in memory. Keeping the > persistent dirty bitmap consistent would be very expensive since it > requires an fdatasync(2) before each write request. > > I think it's reasonable to only write out the in-memory bitmap on > shutdown or live migration. If the QEMU process crashes it is not safe > to trust the dirty bitmap; a full backup must be performed. > > The persistent bitmap file must contain: > 1. Bitmap granularity (e.g. 64 KB) > 2. The actual bitmap (1 TB disk @ 64 KB granularity = 2 MB bitmap) > 3. Flags including a "clean" bit > > When QEMU activates the persistent dirty bitmap, it clears the "clean" > flag. When QEMU deactivates and finishes writing out the dirty bitmap, > it sets the "clean" flag. > > The "clean" flag is used to tell whether the persistent bitmap file is > safe to use again. > > The file contains no information about the VM or disk image. It's up to > the user or management tool to keep track of files.