Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 04/19] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2015-03-16 Thread John Snow
On 03/16/2015 04:44 PM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: The new command pair is added to manage a user created dirty bitmap. The dirty bitmap's name is mandatory and must be unique for the same device, but different devices can have bitmaps with the same names

Re: [Qemu-block] [Qemu-devel] [PATCH] fdc: remove sparc sun4m mutations

2015-03-16 Thread John Snow
On 03/14/2015 12:50 PM, Hervé Poussineau wrote: They were introduced in 6f7e9aec5eb5bdfa57a9e458e391b785c283a007 and 82407d1a4035e5bfefb53ffdcb270872f813b34c and lots of bug fixes were done after that. This fixes (at least) the detection of the floppy controller on Debian 4.0r9/SPARC, and

Re: [Qemu-block] [Qemu-devel] [PATCH 11/11] iotests: 124 - transactional failure test

2015-03-17 Thread John Snow
On 03/17/2015 04:59 PM, Max Reitz wrote: On 2015-03-04 at 23:15, John Snow wrote: Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure

Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 15/19] block: Resize bitmaps on bdrv_truncate

2015-03-17 Thread John Snow
On 03/17/2015 09:50 AM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: Signed-off-by: John Snow js...@redhat.com --- block.c| 18 + include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 52

Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 16/19] hbitmap: truncate tests

2015-03-17 Thread John Snow
On 03/17/2015 10:53 AM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: The general approach is to set bits close to the boundaries of where we are truncating and ensure that everything appears to have gone OK. We test growing and shrinking by different amounts: - Less than

[Qemu-block] [PATCH for-2.3 1/4] ide: fix cmd_write_pio when nsectors 1

2015-03-19 Thread John Snow
We need to adjust the sector being written to prior to calling ide_transfer_start, otherwise we'll write to the same sector again. Signed-off-by: John Snow js...@redhat.com --- hw/ide/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index

[Qemu-block] [PATCH for-2.3 4/4] ahci-test: improve rw buffer patterns

2015-03-19 Thread John Snow
My pattern was cyclical every 256 bytes, so it missed a fairly obvious failure case. Add some rand() pepper into the test pattern, and for large patterns that exceed 256 sectors, start writing an ID per-sector so that we never generate identical sector patterns. Signed-off-by: John Snow js

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/2] ahci: test varying sector offsets

2015-03-24 Thread John Snow
, John Snow wrote: This is a re-send of patches 7 8 from an earlier series, [PATCH v2 0/8] ahci: add more IO tests which ultimately got bounced back because I used some glib functions that were too new. v2: - Patchew caught a pathing problem with the qemu-img binary; the relative path produced

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] ahci: rerror/werror=stop resume tests

2015-03-25 Thread John Snow
On 03/10/2015 04:14 PM, John Snow wrote: This series is based on: [Qemu-devel] [PATCH 0/2] ahci: test varying sector offsets There appear to be some upstream issues for iotests 051 and 061, but this series does not appear to alter the existing bad behavior of those tests. This patchset brings

Re: [Qemu-block] [Qemu-devel] [PATCH RFC for-2.3? 5/8] fdb: Move FDCtrlISABus to header

2015-03-30 Thread John Snow
You probably meant 'fdc' ! On 03/29/2015 01:53 PM, Andreas Färber wrote: To be used for embedding the device. Add gtk-doc private/public markers for parent field. Signed-off-by: Andreas Färber afaer...@suse.de --- hw/block/fdc.c | 87 -

[Qemu-block] qemu-img behavior for locating backing files

2015-04-01 Thread John Snow
Kevin, what's the correct behavior for qemu-img and relative paths when creating a new qcow2 file? Example: (in e.g. /home/qemu/build/ or anywhere not /home: ) qemu-img create -f qcow2 base.qcow2 32G qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2 In 1.7.0., this produces

[Qemu-block] [PATCH v2 09/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-03-27 Thread John Snow
We'd like to be able to specify the callback given to backup_start manually in the case of transactions, so split apart qmp_drive_backup into an implementation and a wrapper. Switch drive_backup_prepare to use the new wrapper, but don't overload the callback and closure yet. Signed-off-by: John

[Qemu-block] [PATCH v2 11/11] iotests: 124 - transactional failure test

2015-03-27 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 119

[Qemu-block] [PATCH v2 01/11] qapi: Add transaction support to block-dirty-bitmap operations

2015-03-27 Thread John Snow
alongside a full backup to accomplish a clean synchronization point. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com --- blockdev.c | 100 +++ qapi-schema.json | 6 +++- 2 files changed, 105 insertions(+), 1

[Qemu-block] [PATCH v2 00/11] block: incremental backup transactions

2015-03-27 Thread John Snow
interest to libvirt, in particular, but only as a convenience feature. Thank you, --John Snow v2: Key: [] : patches are identical [] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C

[Qemu-block] [PATCH v2 08/11] block: move transactions beneath qmp interfaces

2015-03-27 Thread John Snow
In general, since transactions may reference QMP function helpers, it would be nice for them to sit beneath them. This will avoid the need for forward declaring any QMP interfaces, which would be aggravating to update in so many places. Signed-off-by: John Snow js...@redhat.com --- blockdev.c

Re: [Qemu-block] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-23 Thread John Snow
On 04/23/2015 09:19 AM, Stefan Hajnoczi wrote: On Fri, Apr 17, 2015 at 07:49:48PM -0400, John Snow wrote: === v6: === 01: s/underlaying/underlying/ Removed a reference to 'disabled' bitmaps. Touching up inconsistent list indentation. Added FreeBSD Documentation License

Re: [Qemu-block] [Qemu-devel] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-23 Thread John Snow
On 04/23/2015 03:18 PM, Eric Blake wrote: On 04/23/2015 08:41 AM, John Snow wrote: I know I said primarily to be difficult but I was just being facetious. I didn't find the GPL2+ to be suitable for documentation, strictly, so I went to read up on the documentation licenses that the fsf

[Qemu-block] [PATCH v3 10/10] iotests: 124 - transactional failure test

2015-04-22 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 120

[Qemu-block] [PATCH v3 05/10] block: add transactional callbacks feature

2015-04-22 Thread John Snow
-off-by: John Snow js...@redhat.com --- blockdev.c | 183 +++-- 1 file changed, 179 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 2ab63ed..31ccb1b 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1240,6 +1240,8

[Qemu-block] [PATCH v3 06/10] block: add refcount to Job object

2015-04-22 Thread John Snow
If we want to get at the job after the life of the job, we'll need a refcount for this object. This may occur for example if we wish to inspect the actions taken by a particular job after a transactional group of jobs runs, and further actions are required. Signed-off-by: John Snow js

[Qemu-block] [PATCH v3 02/10] iotests: add transactional incremental backup test

2015-04-22 Thread John Snow
Test simple usage cases for using transactions to create and synchronize incremental backups. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 54 ++ tests/qemu-iotests/124.out | 4 ++-- 2 files changed, 56 insertions(+), 2

[Qemu-block] [PATCH v3 07/10] block: add delayed bitmap successor cleanup

2015-04-22 Thread John Snow
to backup_complete. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- block.c | 65 ++- block/backup.c| 20 ++-- include/block/block.h | 10 3 files changed, 70 insertions(+), 25

[Qemu-block] [PATCH v3 7/9] qtest/ahci: add flush migrate test

2015-04-30 Thread John Snow
Use blkdebug to inject an error on first flush, then attempt to flush on the first guest. When the error halts the VM, migrate to the second VM, and attempt to resume the command. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 52

[Qemu-block] [PATCH v3 6/9] qtest/ahci: add migrate dma test

2015-04-30 Thread John Snow
Write to one guest, migrate, and then read from the other. adjust ahci_io to clear any buffers it creates, so that we can use ahci_io safely on both guests knowing we are using empty buffers and not accidentally re-using data. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c

[Qemu-block] [PATCH v3 8/9] qtest/ahci: add halted dma test

2015-04-30 Thread John Snow
If we're going to test the migration of halted DMA jobs, we should probably check to make sure we can resume them locally as a first step. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 60 +++ 1 file changed, 60 insertions

[Qemu-block] [PATCH v3 1/9] libqos/ahci: Add halted command helpers

2015-04-30 Thread John Snow
Sometimes we want a command to halt the VM instead of complete successfully, so it'd be nice to let the libqos/ahci functions cope with such scenarios. Signed-off-by: John Snow js...@redhat.com --- tests/libqos/ahci.c | 27 +++ tests/libqos/ahci.h | 3 +++ 2 files

[Qemu-block] [PATCH v3 0/9] ahci: enable migration

2015-04-30 Thread John Snow
/releases/tag/ahci-migration-test-v3 == John Snow (9): libqos/ahci: Add halted command helpers libqos/ahci: Fix sector set method libqos: Add migration helpers ich9/ahci: Enable Migration qtest/ahci: Add migration test qtest/ahci: add migrate dma test qtest/ahci: add flush migrate test

[Qemu-block] [PATCH v3 2/9] libqos/ahci: Fix sector set method

2015-04-30 Thread John Snow
|| probably does not mean the same thing as |. Additionally, allow users to submit a prd_size of 0 to indicate that they'd like to continue using the default. Signed-off-by: John Snow js...@redhat.com --- tests/libqos/ahci.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff

[Qemu-block] [PATCH v3 4/9] ich9/ahci: Enable Migration

2015-04-30 Thread John Snow
Lift the flag preventing the migration of the ICH9/AHCI devices. Signed-off-by: John Snow js...@redhat.com --- hw/ide/ahci.c | 1 - hw/ide/ich.c | 1 - 2 files changed, 2 deletions(-) diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c index 833fd45..8e36dec 100644 --- a/hw/ide/ahci.c +++ b/hw/ide

[Qemu-block] [PATCH v3 9/9] qtest/ahci: add migrate halted dma test

2015-04-30 Thread John Snow
Test migrating a halted DMA transaction. Resume, then test data integrity. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 75 ++- 1 file changed, 74 insertions(+), 1 deletion(-) diff --git a/tests/ahci-test.c b/tests/ahci

Re: [Qemu-block] [PATCH v3 0/9] ahci: enable migration

2015-05-04 Thread John Snow
On 05/04/2015 08:29 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: The day we all feared is here, and I am proposing we allow the migration of the AHCI device tentatively for the 2.4 development window. There are some more NCQ migration tests are needed, but I felt

Re: [Qemu-block] [Qemu-devel] [PATCH v3 3/9] libqos: Add migration helpers

2015-05-04 Thread John Snow
On 05/04/2015 08:07 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: libqos.c: -set_context for addressing which commands go where -migrate performs the actual migration malloc.c: - Structure of the allocator is adjusted slightly with a second

Re: [Qemu-block] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-01 Thread John Snow
On 04/03/2015 07:05 AM, Paolo Bonzini wrote: On 03/04/2015 12:01, Wen Congyang wrote: Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com --- include/qemu/hbitmap.h | 8

[Qemu-block] [RFC] Differential Backups

2015-04-29 Thread John Snow
This is a feature that should be very easy to add on top of the existing incremental feature, since it's just a difference in how the bitmap is treated: Incremental - Links to the last incremental (managed by libvirt) - Clears the bitmap after creation Differential: - Links to the last full

Re: [Qemu-block] [Qemu-devel] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-07 Thread John Snow
On 05/06/2015 10:20 PM, Wen Congyang wrote: On 05/02/2015 12:47 AM, John Snow wrote: On 04/03/2015 07:05 AM, Paolo Bonzini wrote: On 03/04/2015 12:01, Wen Congyang wrote: Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/9] ahci: enable migration

2015-05-05 Thread John Snow
On 04/30/2015 02:07 PM, John Snow wrote: The day we all feared is here, and I am proposing we allow the migration of the AHCI device tentatively for the 2.4 development window. There are some more NCQ migration tests are needed, but I felt that it was important to get migration enabled

Re: [Qemu-block] [Qemu-devel] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-08 Thread John Snow
On 05/08/2015 09:17 AM, Max Reitz wrote: On 08.05.2015 15:14, Stefan Hajnoczi wrote: On Thu, May 07, 2015 at 01:22:26PM -0400, John Snow wrote: On 05/07/2015 10:54 AM, Stefan Hajnoczi wrote: On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: +static void

Re: [Qemu-block] [Qemu-devel] [PATCH v3 3/9] libqos: Add migration helpers

2015-05-05 Thread John Snow
On 05/05/2015 07:35 AM, Kevin Wolf wrote: Am 04.05.2015 um 19:52 hat John Snow geschrieben: On 05/04/2015 08:07 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: +/* Otherwise, we need to wait: poll until migration is completed. */ +while (1) { +rsp

Re: [Qemu-block] [Qemu-devel] [RFC] Differential Backups

2015-05-05 Thread John Snow
On 05/05/2015 06:25 AM, Stefan Hajnoczi wrote: On Wed, Apr 29, 2015 at 06:51:08PM -0400, John Snow wrote: This is a feature that should be very easy to add on top of the existing incremental feature, since it's just a difference in how the bitmap is treated: Incremental - Links to the last

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/6] block: Fix dirty bitmap in bdrv_co_discard

2015-05-11 Thread John Snow
On 05/06/2015 12:52 AM, Fam Zheng wrote: Unsetting dirty globally with discard is not very correct. The discard may zero out sectors (depending on can_write_zeroes_with_unmap), we should replicate this change to destinition side to make sure that the guest sees the same data. Calling

Re: [Qemu-block] [Qemu-devel] [PATCH v2 5/6] qemu-iotests: Add test case for mirror with unmap

2015-05-11 Thread John Snow
+128,4 @@ 128 rw auto quick 129 rw auto quick 130 rw auto quick +131 rw auto quick Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v2 4/6] qemu-iotests: Make block job methods common

2015-05-11 Thread John Snow
= self.wait_until_completed(drive=drive) +self.assert_qmp(event, 'data/type', 'mirror') + def notrun(reason): '''Skip this test suite''' # Each test in qemu-iotests has a number (seq) Reviewed-by: John Snow js...@redhat.com Side-note: we should at some point clean up the images

Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/6] iotests: Use event_wait in wait_ready

2015-05-11 Thread John Snow
On 05/06/2015 12:52 AM, Fam Zheng wrote: Only poll the specific type of event we are interested in, to avoid stealing events that should be consumed by someone else. Suggested-by: John Snow js...@redhat.com Signed-off-by: Fam Zheng f...@redhat.com --- tests/qemu-iotests/iotests.py | 9

[Qemu-block] [PATCH v4 05/11] block: add transactional callbacks feature

2015-05-11 Thread John Snow
-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- blockdev.c | 183 +++-- 1 file changed, 179 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 068eccb..27db1b4 100644 --- a/blockdev.c

[Qemu-block] [PATCH v4 06/11] block: add refcount to Job object

2015-05-11 Thread John Snow
If we want to get at the job after the life of the job, we'll need a refcount for this object. This may occur for example if we wish to inspect the actions taken by a particular job after a transactional group of jobs runs, and further actions are required. Signed-off-by: John Snow js

[Qemu-block] [PATCH v4 09/11] block: drive_backup transaction callback support

2015-05-11 Thread John Snow
. (4) backup_transaction_complete will perform the final cleanup on the backup job. (5) In the case of transaction cancellation, drive_backup_cb is still responsible for cleaning up the mess we may have already made. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre

[Qemu-block] [PATCH v4 01/11] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-11 Thread John Snow
alongside a full backup to accomplish a clean synchronization point. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com --- block.c | 19 +++- blockdev.c| 114 +- docs/bitmaps.md

[Qemu-block] [PATCH v4 07/11] block: add delayed bitmap successor cleanup

2015-05-11 Thread John Snow
to backup_complete. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- block.c | 65 ++- block/backup.c| 20 ++-- include/block/block.h | 10 3 files changed, 70 insertions(+), 25

[Qemu-block] [PATCH v4 04/11] block: re-add BlkTransactionState

2015-05-11 Thread John Snow
us more efficiently delete items in arbitrary order, which will be more important in the future when some actions will expire at the end of the transaction, but others may persist until all callbacks triggered by the transaction are recollected. Signed-off-by: John Snow js...@redhat.com Reviewed

[Qemu-block] [PATCH v4 02/11] iotests: add transactional incremental backup test

2015-05-11 Thread John Snow
Test simple usage cases for using transactions to create and synchronize incremental backups. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/qemu-iotests/124 | 54

[Qemu-block] [PATCH v4 10/11] iotests: 124 - transactional failure test

2015-05-11 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

[Qemu-block] [PATCH v4 11/11] qmp-commands.hx: Update the supported 'transaction' operations

2015-05-11 Thread John Snow
-snapshot-internal-sync abort block-dirty-bitmap-add block-dirty-bitmap-clear Signed-off-by: Kashyap Chamarthy kcham...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Signed-off-by: John Snow js...@redhat.com --- qmp-commands.hx | 13 - 1 file changed, 8 insertions(+), 5

[Qemu-block] [PATCH v4 08/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-05-11 Thread John Snow
We'd like to be able to specify the callback given to backup_start manually in the case of transactions, so split apart qmp_drive_backup into an implementation and a wrapper. Switch drive_backup_prepare to use the new wrapper, but don't overload the callback and closure yet. Signed-off-by: John

Re: [Qemu-block] [Qemu-devel] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-07 Thread John Snow
On 05/07/2015 10:54 AM, Stefan Hajnoczi wrote: On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: +static void block_dirty_bitmap_clear_prepare(BlkTransactionState *common, + Error **errp) +{ +BlockDirtyBitmapState *state = DO_UPCAST

Re: [Qemu-block] [Qemu-devel] [PATCH 5/5] tests: add test case for encrypted qcow2 read/write

2015-05-12 Thread John Snow
On 05/12/2015 03:52 PM, Eric Blake wrote: On 05/12/2015 01:06 PM, John Snow wrote: tests/qemu-iotests/131 | 69 ++ tests/qemu-iotests/131.out | 46 +++ Fam Zheng already has a patch on-list that uses test 131

Re: [Qemu-block] [PATCH v4 08/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-05-18 Thread John Snow
On 05/18/2015 10:42 AM, Stefan Hajnoczi wrote: On Mon, May 11, 2015 at 07:04:23PM -0400, John Snow wrote: @@ -2900,9 +2917,16 @@ void qmp_drive_backup(const char *device, const char *target, } } +/* If we are not supplied with callback override info, use our defaults */ +if (cb

Re: [Qemu-block] [PATCH v4 09/11] block: drive_backup transaction callback support

2015-05-18 Thread John Snow
On 05/18/2015 11:35 AM, Stefan Hajnoczi wrote: On Mon, May 11, 2015 at 07:04:24PM -0400, John Snow wrote: +static void drive_backup_cb(BlkActionState *common) +{ + BlkActionCallbackData *cb_data = common-cb_data; + BlockDriverState *bs = cb_data-opaque; +DriveBackupState *state

[Qemu-block] [PATCH v5 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-08 Thread John Snow
For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top sync mode. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js

[Qemu-block] [PATCH v5 15/21] block: Resize bitmaps on bdrv_truncate

2015-04-08 Thread John Snow
Signed-off-by: John Snow js...@redhat.com --- block.c| 18 ++ include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 48 3 files changed, 76 insertions(+) diff --git a/block.c b/block.c index 16209a2

[Qemu-block] [PATCH v5 14/21] block: Ensure consistent bitmap function prototypes

2015-04-08 Thread John Snow
We often don't need the BlockDriverState for functions that operate on bitmaps. Remove it. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block.c | 13 ++--- block/backup.c| 2

[Qemu-block] [PATCH v5 01/21] docs: incremental backup documentation

2015-04-08 Thread John Snow
Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow js...@redhat.com --- docs/bitmaps.md | 311 1 file changed, 311 insertions(+) create mode 100644 docs/bitmaps.md diff --git a/docs/bitmaps.md b/docs/bitmaps.md new file

[Qemu-block] [PATCH v5 08/21] block: Add bitmap disabled status

2015-04-08 Thread John Snow
removed for now until a use case emerges where this state must be revealed to the user. The disabled state WILL be used internally for bitmap migration and bitmap persistence. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v5 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-17 Thread John Snow
On 04/17/2015 09:17 AM, Max Reitz wrote: On 09.04.2015 00:19, John Snow wrote: For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top

Re: [Qemu-block] [PATCH v5 01/21] docs: incremental backup documentation

2015-04-17 Thread John Snow
On 04/17/2015 11:06 AM, Eric Blake wrote: On 04/08/2015 04:19 PM, John Snow wrote: Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow js...@redhat.com --- docs/bitmaps.md | 311 1 file changed, 311 insertions

[Qemu-block] [PATCH v6 06/21] hbitmap: cache array lengths

2015-04-17 Thread John Snow
ebl...@redhat.com Signed-off-by: John Snow js...@redhat.com --- util/hbitmap.c | 4 1 file changed, 4 insertions(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index ab13971..5b78613 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -90,6 +90,9 @@ struct HBitmap { * bitmap will still

[Qemu-block] [PATCH v6 05/21] block: Introduce bdrv_dirty_bitmap_granularity()

2015-04-17 Thread John Snow
This returns the granularity (in bytes) of dirty bitmap, which matches the QMP interface and the existing query interface. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com

[Qemu-block] [PATCH v6 11/21] qmp: add block-dirty-bitmap-clear

2015-04-17 Thread John Snow
Add bdrv_clear_dirty_bitmap and a matching QMP command, qmp_block_dirty_bitmap_clear that enables a user to reset the bitmap attached to a drive. This allows us to reset a bitmap in the event of a full drive backup. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre

[Qemu-block] [PATCH v6 07/21] hbitmap: add hbitmap_merge

2015-04-17 Thread John Snow
running speed for particularly sparse bitmaps by using iterators, but the running time for dense maps will be worse. We present the simpler solution first, and we can refine it later if needed. Signed-off-by: John Snow js...@redhat.com --- include/qemu/hbitmap.h | 13 + util

[Qemu-block] [PATCH v6 16/21] hbitmap: truncate tests

2015-04-17 Thread John Snow
long) - Less than sizeof(unsigned long), but across a ulong boundary - More than sizeof(unsigned long) Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/test-hbitmap.c | 255

[Qemu-block] [PATCH v6 19/21] iotests: add simple incremental backup case

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- tests/qemu-iotests/124 | 174 +++-- tests/qemu-iotests/124.out | 4 +- 2 files changed, 172 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/124 b

[Qemu-block] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-17 Thread John Snow
.) Fam Zheng (1): qapi: Add optional field name to block dirty bitmap John Snow (20): docs: incremental backup documentation qmp: Ensure consistent granularity type qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove block: Introduce bdrv_dirty_bitmap_granularity() hbitmap: cache

[Qemu-block] [PATCH v6 15/21] block: Resize bitmaps on bdrv_truncate

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com --- block.c| 18 ++ include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 48 3 files changed, 76 insertions(+) diff --git a/block.c b/block.c index 735acff

[Qemu-block] [PATCH v6 08/21] block: Add bitmap disabled status

2015-04-17 Thread John Snow
removed for now until a use case emerges where this state must be revealed to the user. The disabled state WILL be used internally for bitmap migration and bitmap persistence. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

[Qemu-block] [PATCH v6 17/21] iotests: add invalid input incremental backup tests

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/qemu-iotests/124 | 104 + tests/qemu-iotests/124.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files

[Qemu-block] [PATCH v6 21/21] iotests: add incremental backup granularity tests

2015-04-17 Thread John Snow
Test what happens if you fiddle with the granularity. Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 58 +- tests/qemu-iotests/124.out | 4 ++-- 2 files changed, 49 insertions(+), 13

[Qemu-block] [PATCH v6 13/21] block: add BdrvDirtyBitmap documentation

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block.c b/block.c index ca73a6a..30c8568 100644 --- a/block.c +++ b

[Qemu-block] [PATCH v6 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-17 Thread John Snow
For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top sync mode. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js

Re: [Qemu-block] [Qemu-devel] [PATCH v4 19/20] iotests: add simple incremental backup case

2015-04-06 Thread John Snow
On 04/02/2015 10:27 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:17:02PM -0400, John Snow wrote: Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 153 + tests/qemu-iotests/124.out | 4 +- 2 files changed, 155

Re: [Qemu-block] [Qemu-devel] qemu-img behavior for locating backing files

2015-04-07 Thread John Snow
On 04/07/2015 04:44 AM, Kevin Wolf wrote: Am 07.04.2015 um 02:31 hat John Snow geschrieben: On 04/02/2015 05:38 AM, Kevin Wolf wrote: Am 01.04.2015 um 18:16 hat John Snow geschrieben: Kevin, what's the correct behavior for qemu-img and relative paths when creating a new qcow2 file

Re: [Qemu-block] [PATCH v4 15/20] block: Resize bitmaps on bdrv_truncate

2015-04-07 Thread John Snow
On 04/07/2015 08:57 AM, Stefan Hajnoczi wrote: On Thu, Apr 02, 2015 at 11:57:59AM -0400, John Snow wrote: On 04/02/2015 09:37 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:16:58PM -0400, John Snow wrote: +void hbitmap_truncate(HBitmap *hb, uint64_t size) +{ +bool shrink

Re: [Qemu-block] [Qemu-devel] Migration sometimes fails with IDE and Qemu 2.2.1

2015-04-07 Thread John Snow
On 04/07/2015 02:44 PM, Peter Lieven wrote: Am 07.04.2015 um 17:29 schrieb Dr. David Alan Gilbert: * Peter Lieven (p...@kamp.de) wrote: Hi David, Am 07.04.2015 um 10:43 schrieb Dr. David Alan Gilbert: Any particular workload or reproducer? Workload is almost zero. I try to figure out if

Re: [Qemu-block] [PATCH v4 10/20] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-07 Thread John Snow
On 04/02/2015 08:44 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:16:53PM -0400, John Snow wrote: +} else if (job-sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) { +/* Dirty Bitmap sync has a slightly different iteration method */ +HBitmapIter hbi; +int64_t

Re: [Qemu-block] [PATCH v4 15/20] block: Resize bitmaps on bdrv_truncate

2015-04-02 Thread John Snow
On 04/02/2015 09:37 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:16:58PM -0400, John Snow wrote: +void hbitmap_truncate(HBitmap *hb, uint64_t size) +{ +bool shrink; +unsigned i; +uint64_t num_elements = size; +uint64_t old; + +/* Size comes in as logical elements

Re: [Qemu-block] [PATCH v4 18/20] iotests: add QMP event waiting queue

2015-04-02 Thread John Snow
On 04/02/2015 09:57 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:17:01PM -0400, John Snow wrote: +# Test if 'match' is a recursive subset of 'event' +def event_match(event, match = None): Not worth respinning but PEP8 says there should be no spaces around the '=' for keyword

Re: [Qemu-block] [PATCH v4 10/20] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-02 Thread John Snow
On 04/02/2015 08:44 AM, Stefan Hajnoczi wrote: On Fri, Mar 20, 2015 at 03:16:53PM -0400, John Snow wrote: +} else if (job-sync_mode == MIRROR_SYNC_MODE_DIRTY_BITMAP) { +/* Dirty Bitmap sync has a slightly different iteration method */ +HBitmapIter hbi; +int64_t

Re: [Qemu-block] [Qemu-devel] [PATCH 6/8] fdc: Disentangle phases in fdctrl_read_data()

2015-05-20 Thread John Snow
On 05/20/2015 04:25 AM, Kevin Wolf wrote: Am 19.05.2015 um 22:40 hat John Snow geschrieben: On 05/19/2015 11:36 AM, Kevin Wolf wrote: This commit makes similar improvements as have already been made to the write function: Instead of relying on a flag in the MSR to distinguish controller

Re: [Qemu-block] [Qemu-devel] [PATCH 3/8] fdc: Introduce fdctrl-phase

2015-05-20 Thread John Snow
On 05/20/2015 05:24 AM, Peter Maydell wrote: On 20 May 2015 at 09:43, Kevin Wolf kw...@redhat.com wrote: Am 20.05.2015 um 10:06 hat Peter Maydell geschrieben: That handles migration, which is good. But I still think that storing the same information in two places in the device state (phase

Re: [Qemu-block] [Qemu-devel] [PATCH 7/8] fdc: Fix MSR.RQM flag

2015-05-20 Thread John Snow
On 05/20/2015 04:14 AM, Kevin Wolf wrote: Am 19.05.2015 um 22:40 hat John Snow geschrieben: On 05/19/2015 11:36 AM, Kevin Wolf wrote: The RQM bit in MSR should be set whenever the guest is supposed to access the FIFO, and it should be cleared in all other cases. This is important so

Re: [Qemu-block] [Qemu-devel] [PATCH v4 06/11] block: add refcount to Job object

2015-05-19 Thread John Snow
On 05/18/2015 11:45 AM, Stefan Hajnoczi wrote: On Mon, May 11, 2015 at 07:04:21PM -0400, John Snow wrote: If we want to get at the job after the life of the job, we'll need a refcount for this object. This may occur for example if we wish to inspect the actions taken by a particular job

Re: [Qemu-block] [Qemu-devel] [PATCH 2/8] fdc: Rename fdctrl_set_fifo() to fdctrl_to_result_phase()

2015-05-19 Thread John Snow
) | GET_CUR_DRV(fdctrl); -fdctrl_set_fifo(fdctrl, 1); +fdctrl_to_result_phase(fdctrl, 1); } } Similar bike-shedding comment here to match patch #1, but that won't stop this: Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 3/8] fdc: Introduce fdctrl-phase

2015-05-19 Thread John Snow
] Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 1/8] fdc: Rename fdctrl_reset_fifo() to fdctrl_to_command_phase()

2015-05-19 Thread John Snow
. Bike-shedding aside: Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 7/8] fdc: Fix MSR.RQM flag

2015-05-19 Thread John Snow
On 05/19/2015 11:36 AM, Kevin Wolf wrote: The RQM bit in MSR should be set whenever the guest is supposed to access the FIFO, and it should be cleared in all other cases. This is important so the guest can't continue writing/reading the FIFO beyond the length that it's suppossed to access

Re: [Qemu-block] [Qemu-devel] [PATCH 5/8] fdc: Code cleanup in fdctrl_write_data()

2015-05-19 Thread John Snow
-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 4/8] fdc: Use phase in fdctrl_write_data()

2015-05-19 Thread John Snow
++ 1 file changed, 39 insertions(+), 28 deletions(-) diff --git a/hw/block/fdc.c b/hw/block/fdc.c index 4d4868e..a13e0ce 100644 --- a/hw/block/fdc.c +++ b/hw/block/fdc.c [snip] Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 6/8] fdc: Disentangle phases in fdctrl_read_data()

2015-05-19 Thread John Snow
); fdctrl_reset_irq(fdctrl); } +break; + +case FD_PHASE_COMMAND: +default: +abort(); } + +retval = fdctrl-fifo[pos]; FLOPPY_DPRINTF(data register: 0x%02x\n, retval); return retval; Reviewed-by: John Snow js...@redhat.com

[Qemu-block] [PATCH 7/9] block: add differential backup mode

2015-06-04 Thread John Snow
This is simple: instead of clearing the bitmap, just leave the bitmap data intact even in case of success. Signed-off-by: John Snow js...@redhat.com --- block.c | 9 - block/backup.c| 17 ++--- block/mirror.c| 9 +++-- include/block/block.h

[Qemu-block] [PATCH 1/9] qapi: Rename 'dirty-bitmap' mode to 'incremental'

2015-06-04 Thread John Snow
this flexibility. Signed-off-by: John Snow js...@redhat.com --- block/backup.c| 10 +- block/mirror.c| 4 ++-- docs/bitmaps.md | 8 include/block/block_int.h | 2 +- qapi/block-core.json | 8 qmp-commands.hx | 6

[Qemu-block] [PATCH 0/9] block: add differential backup support

2015-06-04 Thread John Snow
://github.com/jnsnow/qemu/tree/differential-backup This version is tagged differential-backup-v1: https://github.com/jnsnow/qemu/releases/tag/differential-backup-v1 == John Snow (9): qapi: Rename 'dirty-bitmap' mode to 'incremental' hbitmap: add hbitmap_copy block: add bdrv_copy_dirty_bitmap

  1   2   3   4   5   6   7   8   9   10   >