Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 04/19] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2015-03-16 Thread John Snow
On 03/16/2015 04:44 PM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: The new command pair is added to manage a user created dirty bitmap. The dirty bitmap's name is mandatory and must be unique for the same device, but different devices can have bitmaps with the same names

Re: [Qemu-block] [Qemu-devel] [PATCH] fdc: remove sparc sun4m mutations

2015-03-16 Thread John Snow
On 03/14/2015 12:50 PM, Hervé Poussineau wrote: They were introduced in 6f7e9aec5eb5bdfa57a9e458e391b785c283a007 and 82407d1a4035e5bfefb53ffdcb270872f813b34c and lots of bug fixes were done after that. This fixes (at least) the detection of the floppy controller on Debian 4.0r9/SPARC, and

Re: [Qemu-block] [Qemu-devel] [PATCH 11/11] iotests: 124 - transactional failure test

2015-03-17 Thread John Snow
On 03/17/2015 04:59 PM, Max Reitz wrote: On 2015-03-04 at 23:15, John Snow wrote: Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure

Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 15/19] block: Resize bitmaps on bdrv_truncate

2015-03-17 Thread John Snow
On 03/17/2015 09:50 AM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: Signed-off-by: John Snow js...@redhat.com --- block.c| 18 + include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 52

Re: [Qemu-block] [Qemu-devel] [2.4 PATCH v3 16/19] hbitmap: truncate tests

2015-03-17 Thread John Snow
On 03/17/2015 10:53 AM, Max Reitz wrote: On 2015-03-13 at 14:30, John Snow wrote: The general approach is to set bits close to the boundaries of where we are truncating and ensure that everything appears to have gone OK. We test growing and shrinking by different amounts: - Less than

[Qemu-block] 2.3-rc0 ahci-test failure on ppc64

2015-03-18 Thread John Snow
Andreas brought this failure to my attention; it looks like PIO read/writes for nsectors 1 for IDE or AHCI will fail on ppc64. The failure is in the IDE core layer. In a nutshell, we've never *actually* supported PIO r/w for nsectors 1, I just started testing it as a spec compliance item

[Qemu-block] [PATCH for-2.3 1/4] ide: fix cmd_write_pio when nsectors 1

2015-03-19 Thread John Snow
We need to adjust the sector being written to prior to calling ide_transfer_start, otherwise we'll write to the same sector again. Signed-off-by: John Snow js...@redhat.com --- hw/ide/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index

[Qemu-block] [PATCH for-2.3 4/4] ahci-test: improve rw buffer patterns

2015-03-19 Thread John Snow
My pattern was cyclical every 256 bytes, so it missed a fairly obvious failure case. Add some rand() pepper into the test pattern, and for large patterns that exceed 256 sectors, start writing an ID per-sector so that we never generate identical sector patterns. Signed-off-by: John Snow js

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/2] ahci: test varying sector offsets

2015-03-24 Thread John Snow
, John Snow wrote: This is a re-send of patches 7 8 from an earlier series, [PATCH v2 0/8] ahci: add more IO tests which ultimately got bounced back because I used some glib functions that were too new. v2: - Patchew caught a pathing problem with the qemu-img binary; the relative path produced

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] ahci: rerror/werror=stop resume tests

2015-03-25 Thread John Snow
On 03/10/2015 04:14 PM, John Snow wrote: This series is based on: [Qemu-devel] [PATCH 0/2] ahci: test varying sector offsets There appear to be some upstream issues for iotests 051 and 061, but this series does not appear to alter the existing bad behavior of those tests. This patchset brings

Re: [Qemu-block] [Qemu-devel] [PATCH RFC for-2.3? 5/8] fdb: Move FDCtrlISABus to header

2015-03-30 Thread John Snow
You probably meant 'fdc' ! On 03/29/2015 01:53 PM, Andreas Färber wrote: To be used for embedding the device. Add gtk-doc private/public markers for parent field. Signed-off-by: Andreas Färber afaer...@suse.de --- hw/block/fdc.c | 87 -

[Qemu-block] qemu-img behavior for locating backing files

2015-04-01 Thread John Snow
Kevin, what's the correct behavior for qemu-img and relative paths when creating a new qcow2 file? Example: (in e.g. /home/qemu/build/ or anywhere not /home: ) qemu-img create -f qcow2 base.qcow2 32G qemu-img create -f qcow2 -F qcow2 -b base.qcow2 /home/overlay.qcow2 In 1.7.0., this produces

[Qemu-block] [PATCH v2 04/11] block: re-add BlkTransactionState

2015-03-27 Thread John Snow
us more efficiently delete items in arbitrary order, which will be more important in the future when some actions will expire at the end of the transaction, but others may persist until all callbacks triggered by the transaction are recollected. Signed-off-by: John Snow js...@redhat.com

[Qemu-block] [PATCH v2 09/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-03-27 Thread John Snow
We'd like to be able to specify the callback given to backup_start manually in the case of transactions, so split apart qmp_drive_backup into an implementation and a wrapper. Switch drive_backup_prepare to use the new wrapper, but don't overload the callback and closure yet. Signed-off-by: John

[Qemu-block] [PATCH v2 11/11] iotests: 124 - transactional failure test

2015-03-27 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 119

[Qemu-block] [PATCH v2 01/11] qapi: Add transaction support to block-dirty-bitmap operations

2015-03-27 Thread John Snow
alongside a full backup to accomplish a clean synchronization point. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com --- blockdev.c | 100 +++ qapi-schema.json | 6 +++- 2 files changed, 105 insertions(+), 1

[Qemu-block] [PATCH v2 00/11] block: incremental backup transactions

2015-03-27 Thread John Snow
interest to libvirt, in particular, but only as a convenience feature. Thank you, --John Snow v2: Key: [] : patches are identical [] : number of functional differences between upstream/downstream patch [down] : patch is downstream-only The flags [FC] indicate (F)unctional and (C

[Qemu-block] [PATCH v2 08/11] block: move transactions beneath qmp interfaces

2015-03-27 Thread John Snow
In general, since transactions may reference QMP function helpers, it would be nice for them to sit beneath them. This will avoid the need for forward declaring any QMP interfaces, which would be aggravating to update in so many places. Signed-off-by: John Snow js...@redhat.com --- blockdev.c

Re: [Qemu-block] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-23 Thread John Snow
On 04/23/2015 09:19 AM, Stefan Hajnoczi wrote: On Fri, Apr 17, 2015 at 07:49:48PM -0400, John Snow wrote: === v6: === 01: s/underlaying/underlying/ Removed a reference to 'disabled' bitmaps. Touching up inconsistent list indentation. Added FreeBSD Documentation License

Re: [Qemu-block] [Qemu-devel] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-23 Thread John Snow
On 04/23/2015 03:18 PM, Eric Blake wrote: On 04/23/2015 08:41 AM, John Snow wrote: I know I said primarily to be difficult but I was just being facetious. I didn't find the GPL2+ to be suitable for documentation, strictly, so I went to read up on the documentation licenses that the fsf

[Qemu-block] [PATCH v3 10/10] iotests: 124 - transactional failure test

2015-04-22 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 120

[Qemu-block] [PATCH v3 05/10] block: add transactional callbacks feature

2015-04-22 Thread John Snow
-off-by: John Snow js...@redhat.com --- blockdev.c | 183 +++-- 1 file changed, 179 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 2ab63ed..31ccb1b 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1240,6 +1240,8

[Qemu-block] [PATCH v3 00/10] block: incremental backup transactions

2015-04-22 Thread John Snow
to wait_incremental() in test_transaction_failure() John Snow (10): qapi: Add transaction support to block-dirty-bitmap operations iotests: add transactional incremental backup test block: rename BlkTransactionState and BdrvActionOps block: re-add BlkTransactionState block: add

[Qemu-block] [PATCH v3 06/10] block: add refcount to Job object

2015-04-22 Thread John Snow
If we want to get at the job after the life of the job, we'll need a refcount for this object. This may occur for example if we wish to inspect the actions taken by a particular job after a transactional group of jobs runs, and further actions are required. Signed-off-by: John Snow js

[Qemu-block] [PATCH v3 03/10] block: rename BlkTransactionState and BdrvActionOps

2015-04-22 Thread John Snow
and names, and should not affect behavior in any way. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- blockdev.c | 114 ++--- 1 file changed, 64 insertions(+), 50 deletions(-) diff --git a/blockdev.c b

[Qemu-block] [PATCH v3 04/10] block: re-add BlkTransactionState

2015-04-22 Thread John Snow
us more efficiently delete items in arbitrary order, which will be more important in the future when some actions will expire at the end of the transaction, but others may persist until all callbacks triggered by the transaction are recollected. Signed-off-by: John Snow js...@redhat.com Reviewed

[Qemu-block] [PATCH v3 02/10] iotests: add transactional incremental backup test

2015-04-22 Thread John Snow
Test simple usage cases for using transactions to create and synchronize incremental backups. Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 54 ++ tests/qemu-iotests/124.out | 4 ++-- 2 files changed, 56 insertions(+), 2

[Qemu-block] [PATCH v3 09/10] block: drive_backup transaction callback support

2015-04-22 Thread John Snow
. (4) backup_transaction_complete will perform the final cleanup on the backup job. (5) In the case of transaction cancellation, drive_backup_cb is still responsible for cleaning up the mess we may have already made. Signed-off-by: John Snow js...@redhat.com --- block/backup.c

[Qemu-block] [PATCH v3 07/10] block: add delayed bitmap successor cleanup

2015-04-22 Thread John Snow
to backup_complete. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- block.c | 65 ++- block/backup.c| 20 ++-- include/block/block.h | 10 3 files changed, 70 insertions(+), 25

[Qemu-block] [PATCH v3 08/10] qmp: Add an implementation wrapper for qmp_drive_backup

2015-04-22 Thread John Snow
We'd like to be able to specify the callback given to backup_start manually in the case of transactions, so split apart qmp_drive_backup into an implementation and a wrapper. Switch drive_backup_prepare to use the new wrapper, but don't overload the callback and closure yet. Signed-off-by: John

[Qemu-block] [PATCH v3 7/9] qtest/ahci: add flush migrate test

2015-04-30 Thread John Snow
Use blkdebug to inject an error on first flush, then attempt to flush on the first guest. When the error halts the VM, migrate to the second VM, and attempt to resume the command. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 52

[Qemu-block] [PATCH v3 6/9] qtest/ahci: add migrate dma test

2015-04-30 Thread John Snow
Write to one guest, migrate, and then read from the other. adjust ahci_io to clear any buffers it creates, so that we can use ahci_io safely on both guests knowing we are using empty buffers and not accidentally re-using data. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c

[Qemu-block] [PATCH v3 8/9] qtest/ahci: add halted dma test

2015-04-30 Thread John Snow
If we're going to test the migration of halted DMA jobs, we should probably check to make sure we can resume them locally as a first step. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 60 +++ 1 file changed, 60 insertions

[Qemu-block] [PATCH v3 1/9] libqos/ahci: Add halted command helpers

2015-04-30 Thread John Snow
Sometimes we want a command to halt the VM instead of complete successfully, so it'd be nice to let the libqos/ahci functions cope with such scenarios. Signed-off-by: John Snow js...@redhat.com --- tests/libqos/ahci.c | 27 +++ tests/libqos/ahci.h | 3 +++ 2 files

[Qemu-block] [PATCH v3 0/9] ahci: enable migration

2015-04-30 Thread John Snow
/releases/tag/ahci-migration-test-v3 == John Snow (9): libqos/ahci: Add halted command helpers libqos/ahci: Fix sector set method libqos: Add migration helpers ich9/ahci: Enable Migration qtest/ahci: Add migration test qtest/ahci: add migrate dma test qtest/ahci: add flush migrate test

[Qemu-block] [PATCH v3 3/9] libqos: Add migration helpers

2015-04-30 Thread John Snow
to the destination. Signed-off-by: John Snow js...@redhat.com --- tests/libqos/libqos.c | 84 +++ tests/libqos/libqos.h | 2 ++ tests/libqos/malloc.c | 74 ++--- tests/libqos/malloc.h | 1 + 4 files changed

[Qemu-block] [PATCH v3 2/9] libqos/ahci: Fix sector set method

2015-04-30 Thread John Snow
|| probably does not mean the same thing as |. Additionally, allow users to submit a prd_size of 0 to indicate that they'd like to continue using the default. Signed-off-by: John Snow js...@redhat.com --- tests/libqos/ahci.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff

[Qemu-block] [PATCH v3 4/9] ich9/ahci: Enable Migration

2015-04-30 Thread John Snow
Lift the flag preventing the migration of the ICH9/AHCI devices. Signed-off-by: John Snow js...@redhat.com --- hw/ide/ahci.c | 1 - hw/ide/ich.c | 1 - 2 files changed, 2 deletions(-) diff --git a/hw/ide/ahci.c b/hw/ide/ahci.c index 833fd45..8e36dec 100644 --- a/hw/ide/ahci.c +++ b/hw/ide

[Qemu-block] [PATCH v3 9/9] qtest/ahci: add migrate halted dma test

2015-04-30 Thread John Snow
Test migrating a halted DMA transaction. Resume, then test data integrity. Signed-off-by: John Snow js...@redhat.com --- tests/ahci-test.c | 75 ++- 1 file changed, 74 insertions(+), 1 deletion(-) diff --git a/tests/ahci-test.c b/tests/ahci

Re: [Qemu-block] [PATCH v3 0/9] ahci: enable migration

2015-05-04 Thread John Snow
On 05/04/2015 08:29 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: The day we all feared is here, and I am proposing we allow the migration of the AHCI device tentatively for the 2.4 development window. There are some more NCQ migration tests are needed, but I felt

Re: [Qemu-block] [Qemu-devel] [PATCH v3 3/9] libqos: Add migration helpers

2015-05-04 Thread John Snow
On 05/04/2015 08:07 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: libqos.c: -set_context for addressing which commands go where -migrate performs the actual migration malloc.c: - Structure of the allocator is adjusted slightly with a second

Re: [Qemu-block] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-01 Thread John Snow
On 04/03/2015 07:05 AM, Paolo Bonzini wrote: On 03/04/2015 12:01, Wen Congyang wrote: Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed-off-by: Gonglei arei.gong...@huawei.com --- include/qemu/hbitmap.h | 8

[Qemu-block] [RFC] Differential Backups

2015-04-29 Thread John Snow
This is a feature that should be very easy to add on top of the existing incremental feature, since it's just a difference in how the bitmap is treated: Incremental - Links to the last incremental (managed by libvirt) - Clears the bitmap after creation Differential: - Links to the last full

Re: [Qemu-block] [Qemu-devel] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-07 Thread John Snow
On 05/06/2015 10:20 PM, Wen Congyang wrote: On 05/02/2015 12:47 AM, John Snow wrote: On 04/03/2015 07:05 AM, Paolo Bonzini wrote: On 03/04/2015 12:01, Wen Congyang wrote: Signed-off-by: Wen Congyang we...@cn.fujitsu.com Signed-off-by: zhanghailiang zhang.zhanghaili...@huawei.com Signed

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/9] ahci: enable migration

2015-05-05 Thread John Snow
On 04/30/2015 02:07 PM, John Snow wrote: The day we all feared is here, and I am proposing we allow the migration of the AHCI device tentatively for the 2.4 development window. There are some more NCQ migration tests are needed, but I felt that it was important to get migration enabled

Re: [Qemu-block] [Qemu-devel] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-08 Thread John Snow
On 05/08/2015 09:17 AM, Max Reitz wrote: On 08.05.2015 15:14, Stefan Hajnoczi wrote: On Thu, May 07, 2015 at 01:22:26PM -0400, John Snow wrote: On 05/07/2015 10:54 AM, Stefan Hajnoczi wrote: On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: +static void

Re: [Qemu-block] [Qemu-devel] [PATCH v3 3/9] libqos: Add migration helpers

2015-05-05 Thread John Snow
On 05/05/2015 07:35 AM, Kevin Wolf wrote: Am 04.05.2015 um 19:52 hat John Snow geschrieben: On 05/04/2015 08:07 AM, Kevin Wolf wrote: Am 30.04.2015 um 20:07 hat John Snow geschrieben: +/* Otherwise, we need to wait: poll until migration is completed. */ +while (1) { +rsp

Re: [Qemu-block] [Qemu-devel] [RFC] Differential Backups

2015-05-05 Thread John Snow
On 05/05/2015 06:25 AM, Stefan Hajnoczi wrote: On Wed, Apr 29, 2015 at 06:51:08PM -0400, John Snow wrote: This is a feature that should be very easy to add on top of the existing incremental feature, since it's just a difference in how the bitmap is treated: Incremental - Links to the last

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/6] block: Remove bdrv_reset_dirty

2015-05-11 Thread John Snow
(BlockDriverState *bs, int64_t cur_sector, - int nr_sectors); #endif /* BLOCK_INT_H */ Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/6] block: Fix dirty bitmap in bdrv_co_discard

2015-05-11 Thread John Snow
On 05/06/2015 12:52 AM, Fam Zheng wrote: Unsetting dirty globally with discard is not very correct. The discard may zero out sectors (depending on can_write_zeroes_with_unmap), we should replicate this change to destinition side to make sure that the guest sees the same data. Calling

Re: [Qemu-block] [Qemu-devel] [PATCH v2 5/6] qemu-iotests: Add test case for mirror with unmap

2015-05-11 Thread John Snow
+128,4 @@ 128 rw auto quick 129 rw auto quick 130 rw auto quick +131 rw auto quick Reviewed-by: John Snow js...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v2 4/6] qemu-iotests: Make block job methods common

2015-05-11 Thread John Snow
= self.wait_until_completed(drive=drive) +self.assert_qmp(event, 'data/type', 'mirror') + def notrun(reason): '''Skip this test suite''' # Each test in qemu-iotests has a number (seq) Reviewed-by: John Snow js...@redhat.com Side-note: we should at some point clean up the images

Re: [Qemu-block] [Qemu-devel] [PATCH v2 6/6] iotests: Use event_wait in wait_ready

2015-05-11 Thread John Snow
On 05/06/2015 12:52 AM, Fam Zheng wrote: Only poll the specific type of event we are interested in, to avoid stealing events that should be consumed by someone else. Suggested-by: John Snow js...@redhat.com Signed-off-by: Fam Zheng f...@redhat.com --- tests/qemu-iotests/iotests.py | 9

[Qemu-block] [PATCH v4 05/11] block: add transactional callbacks feature

2015-05-11 Thread John Snow
-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- blockdev.c | 183 +++-- 1 file changed, 179 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c index 068eccb..27db1b4 100644 --- a/blockdev.c

[Qemu-block] [PATCH v4 00/11] block: incremental backup transactions

2015-05-11 Thread John Snow
://github.com/jnsnow/qemu/tree/incremental-transactions This version is tagged incremental-transactions-v4: https://github.com/jnsnow/qemu/releases/tag/incremental-transactions-v4 == John Snow (10): qapi: Add transaction support to block-dirty-bitmap operations iotests: add transactional incremental

[Qemu-block] [PATCH v4 06/11] block: add refcount to Job object

2015-05-11 Thread John Snow
If we want to get at the job after the life of the job, we'll need a refcount for this object. This may occur for example if we wish to inspect the actions taken by a particular job after a transactional group of jobs runs, and further actions are required. Signed-off-by: John Snow js

[Qemu-block] [PATCH v4 09/11] block: drive_backup transaction callback support

2015-05-11 Thread John Snow
. (4) backup_transaction_complete will perform the final cleanup on the backup job. (5) In the case of transaction cancellation, drive_backup_cb is still responsible for cleaning up the mess we may have already made. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre

[Qemu-block] [PATCH v4 01/11] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-11 Thread John Snow
alongside a full backup to accomplish a clean synchronization point. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com --- block.c | 19 +++- blockdev.c| 114 +- docs/bitmaps.md

[Qemu-block] [PATCH v4 07/11] block: add delayed bitmap successor cleanup

2015-05-11 Thread John Snow
to backup_complete. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- block.c | 65 ++- block/backup.c| 20 ++-- include/block/block.h | 10 3 files changed, 70 insertions(+), 25

[Qemu-block] [PATCH v4 04/11] block: re-add BlkTransactionState

2015-05-11 Thread John Snow
us more efficiently delete items in arbitrary order, which will be more important in the future when some actions will expire at the end of the transaction, but others may persist until all callbacks triggered by the transaction are recollected. Signed-off-by: John Snow js...@redhat.com Reviewed

[Qemu-block] [PATCH v4 02/11] iotests: add transactional incremental backup test

2015-05-11 Thread John Snow
Test simple usage cases for using transactions to create and synchronize incremental backups. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/qemu-iotests/124 | 54

[Qemu-block] [PATCH v4 10/11] iotests: 124 - transactional failure test

2015-05-11 Thread John Snow
Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

[Qemu-block] [PATCH v4 11/11] qmp-commands.hx: Update the supported 'transaction' operations

2015-05-11 Thread John Snow
-snapshot-internal-sync abort block-dirty-bitmap-add block-dirty-bitmap-clear Signed-off-by: Kashyap Chamarthy kcham...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Signed-off-by: John Snow js...@redhat.com --- qmp-commands.hx | 13 - 1 file changed, 8 insertions(+), 5

[Qemu-block] [PATCH v4 08/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-05-11 Thread John Snow
We'd like to be able to specify the callback given to backup_start manually in the case of transactions, so split apart qmp_drive_backup into an implementation and a wrapper. Switch drive_backup_prepare to use the new wrapper, but don't overload the callback and closure yet. Signed-off-by: John

Re: [Qemu-block] [Qemu-devel] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-07 Thread John Snow
On 05/07/2015 10:54 AM, Stefan Hajnoczi wrote: On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: +static void block_dirty_bitmap_clear_prepare(BlkTransactionState *common, + Error **errp) +{ +BlockDirtyBitmapState *state = DO_UPCAST

Re: [Qemu-block] [Qemu-devel] [PATCH 5/5] tests: add test case for encrypted qcow2 read/write

2015-05-12 Thread John Snow
On 05/12/2015 03:52 PM, Eric Blake wrote: On 05/12/2015 01:06 PM, John Snow wrote: tests/qemu-iotests/131 | 69 ++ tests/qemu-iotests/131.out | 46 +++ Fam Zheng already has a patch on-list that uses test 131

Re: [Qemu-block] [PATCH v4 08/11] qmp: Add an implementation wrapper for qmp_drive_backup

2015-05-18 Thread John Snow
On 05/18/2015 10:42 AM, Stefan Hajnoczi wrote: On Mon, May 11, 2015 at 07:04:23PM -0400, John Snow wrote: @@ -2900,9 +2917,16 @@ void qmp_drive_backup(const char *device, const char *target, } } +/* If we are not supplied with callback override info, use our defaults */ +if (cb

Re: [Qemu-block] [PATCH v4 09/11] block: drive_backup transaction callback support

2015-05-18 Thread John Snow
On 05/18/2015 11:35 AM, Stefan Hajnoczi wrote: On Mon, May 11, 2015 at 07:04:24PM -0400, John Snow wrote: +static void drive_backup_cb(BlkActionState *common) +{ + BlkActionCallbackData *cb_data = common-cb_data; + BlockDriverState *bs = cb_data-opaque; +DriveBackupState *state

Re: [Qemu-block] [Qemu-devel] [PATCH] qemu-iotests: Make debugging python tests easier

2015-05-15 Thread John Snow
On 05/15/2015 01:26 AM, Fam Zheng wrote: Adding -d option. The output goes to tee so it appears in your console. Also, raise the verbosity of unnitest runner. When testing a topic branch, it's possible that a bug introduced by a code change makes the python test case hang, with debug

[Qemu-block] [PATCH v5 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-08 Thread John Snow
For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top sync mode. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js

[Qemu-block] [PATCH v5 06/21] hbitmap: cache array lengths

2015-04-08 Thread John Snow
As a convenience: between incremental backups, bitmap migrations and bitmap persistence we seem to need to recalculate these a lot. Because the lengths are a little bit-twiddly, let's just solidly cache them and be done with it. Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow

[Qemu-block] [PATCH v5 07/21] hbitmap: add hbitmap_merge

2015-04-08 Thread John Snow
running speed for particularly sparse bitmaps by using iterators, but the running time for dense maps will be worse. We present the simpler solution first, and we can refine it later if needed. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan

[Qemu-block] [PATCH v5 00/21] block: transactionless incremental backup series

2015-04-08 Thread John Snow
QMP command documentation. 11: Since 2.4 15: Test 112 -- 124 17: Number of tests altered. (Only 4, now.) Fam Zheng (1): qapi: Add optional field name to block dirty bitmap John Snow (20): docs: incremental backup documentation qmp: Ensure consistent granularity type qmp: Add block-dirty

[Qemu-block] [PATCH v5 19/21] iotests: add simple incremental backup case

2015-04-08 Thread John Snow
Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 174 +++-- tests/qemu-iotests/124.out | 4 +- 2 files changed, 172 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124 index 85675ec

[Qemu-block] [PATCH v5 15/21] block: Resize bitmaps on bdrv_truncate

2015-04-08 Thread John Snow
Signed-off-by: John Snow js...@redhat.com --- block.c| 18 ++ include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 48 3 files changed, 76 insertions(+) diff --git a/block.c b/block.c index 16209a2

[Qemu-block] [PATCH v5 14/21] block: Ensure consistent bitmap function prototypes

2015-04-08 Thread John Snow
We often don't need the BlockDriverState for functions that operate on bitmaps. Remove it. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block.c | 13 ++--- block/backup.c| 2

[Qemu-block] [PATCH v5 01/21] docs: incremental backup documentation

2015-04-08 Thread John Snow
Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow js...@redhat.com --- docs/bitmaps.md | 311 1 file changed, 311 insertions(+) create mode 100644 docs/bitmaps.md diff --git a/docs/bitmaps.md b/docs/bitmaps.md new file

[Qemu-block] [PATCH v5 18/21] iotests: add QMP event waiting queue

2015-04-08 Thread John Snow
pull_event method's wait parameter to allow tests to fail if they do not complete within some expected period of time. Signed-off-by: John Snow js...@redhat.com --- scripts/qmp/qmp.py| 6 +- tests/qemu-iotests/iotests.py | 38 ++ 2 files changed

[Qemu-block] [PATCH v5 04/21] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2015-04-08 Thread John Snow
will be re-used in future patches in this series, see: 'qapi: Add transaction support to block-dirty-bitmap-{add, enable, disable}' Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block.c | 20

[Qemu-block] [PATCH v5 17/21] iotests: add invalid input incremental backup tests

2015-04-08 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/qemu-iotests/124 | 104 + tests/qemu-iotests/124.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files

[Qemu-block] [PATCH v5 08/21] block: Add bitmap disabled status

2015-04-08 Thread John Snow
removed for now until a use case emerges where this state must be revealed to the user. The disabled state WILL be used internally for bitmap migration and bitmap persistence. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH v5 18/21] iotests: add QMP event waiting queue

2015-04-17 Thread John Snow
On 04/17/2015 09:33 AM, Max Reitz wrote: On 09.04.2015 00:20, John Snow wrote: A filter is added to allow callers to request very specific events to be pulled from the event queue, while leaving undesired events still in the stream. This allows to poll for completion data for multiple

Re: [Qemu-block] [Qemu-devel] [PATCH v5 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-17 Thread John Snow
On 04/17/2015 09:17 AM, Max Reitz wrote: On 09.04.2015 00:19, John Snow wrote: For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top

Re: [Qemu-block] [Qemu-devel] [PATCH v5 19/21] iotests: add simple incremental backup case

2015-04-17 Thread John Snow
On 04/17/2015 10:33 AM, Max Reitz wrote: On 09.04.2015 00:20, John Snow wrote: Signed-off-by: John Snow js...@redhat.com --- tests/qemu-iotests/124 | 174 +++-- tests/qemu-iotests/124.out | 4 +- 2 files changed, 172 insertions(+), 6 deletions

Re: [Qemu-block] [PATCH v5 01/21] docs: incremental backup documentation

2015-04-17 Thread John Snow
On 04/17/2015 11:06 AM, Eric Blake wrote: On 04/08/2015 04:19 PM, John Snow wrote: Reviewed-by: Max Reitz mre...@redhat.com Signed-off-by: John Snow js...@redhat.com --- docs/bitmaps.md | 311 1 file changed, 311 insertions

[Qemu-block] [PATCH v6 06/21] hbitmap: cache array lengths

2015-04-17 Thread John Snow
ebl...@redhat.com Signed-off-by: John Snow js...@redhat.com --- util/hbitmap.c | 4 1 file changed, 4 insertions(+) diff --git a/util/hbitmap.c b/util/hbitmap.c index ab13971..5b78613 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -90,6 +90,9 @@ struct HBitmap { * bitmap will still

[Qemu-block] [PATCH v6 05/21] block: Introduce bdrv_dirty_bitmap_granularity()

2015-04-17 Thread John Snow
This returns the granularity (in bytes) of dirty bitmap, which matches the QMP interface and the existing query interface. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com

[Qemu-block] [PATCH v6 03/21] qmp: Ensure consistent granularity type

2015-04-17 Thread John Snow
We treat this field with a variety of different types everywhere in the code. Now it's just uint32_t. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- block.c

Re: [Qemu-block] [Qemu-devel] [PATCH v2 05/11] block: add transactional callbacks feature

2015-04-17 Thread John Snow
On 04/17/2015 11:41 AM, Max Reitz wrote: On 27.03.2015 20:19, John Snow wrote: The goal here is to add a new method to transactions that allows developers to specify a callback that will get invoked only once all jobs spawned by a transaction are completed, allowing developers the chance

[Qemu-block] [PATCH v6 04/21] qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove

2015-04-17 Thread John Snow
will be re-used in future patches in this series, see: 'qapi: Add transaction support to block-dirty-bitmap-{add, enable, disable}' Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com

[Qemu-block] [PATCH v6 11/21] qmp: add block-dirty-bitmap-clear

2015-04-17 Thread John Snow
Add bdrv_clear_dirty_bitmap and a matching QMP command, qmp_block_dirty_bitmap_clear that enables a user to reset the bitmap attached to a drive. This allows us to reset a bitmap in the event of a full drive backup. Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre

[Qemu-block] [PATCH v6 07/21] hbitmap: add hbitmap_merge

2015-04-17 Thread John Snow
running speed for particularly sparse bitmaps by using iterators, but the running time for dense maps will be worse. We present the simpler solution first, and we can refine it later if needed. Signed-off-by: John Snow js...@redhat.com --- include/qemu/hbitmap.h | 13 + util

[Qemu-block] [PATCH v6 16/21] hbitmap: truncate tests

2015-04-17 Thread John Snow
long) - Less than sizeof(unsigned long), but across a ulong boundary - More than sizeof(unsigned long) Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/test-hbitmap.c | 255

[Qemu-block] [PATCH v6 19/21] iotests: add simple incremental backup case

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- tests/qemu-iotests/124 | 174 +++-- tests/qemu-iotests/124.out | 4 +- 2 files changed, 172 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/124 b

Re: [Qemu-block] [PATCH v5 10/21] qmp: Add support of dirty-bitmap sync mode for drive-backup

2015-04-17 Thread John Snow
On 04/17/2015 06:51 PM, Eric Blake wrote: On 04/08/2015 04:19 PM, John Snow wrote: For dirty-bitmap sync mode, the block job will iterate through the given dirty bitmap to decide if a sector needs backup (backup all the dirty clusters and skip clean ones), just as allocation conditions of top

[Qemu-block] [PATCH v6 02/21] qapi: Add optional field name to block dirty bitmap

2015-04-17 Thread John Snow
name. Implemented bdrv_find_dirty_bitmap to find a dirty bitmap by name, will be used later when other QMP commands want to reference dirty bitmap by name. Add bdrv_dirty_bitmap_make_anon. This unsets the name of dirty bitmap. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js

[Qemu-block] [PATCH v6 00/21] block: transactionless incremental backup series

2015-04-17 Thread John Snow
.) Fam Zheng (1): qapi: Add optional field name to block dirty bitmap John Snow (20): docs: incremental backup documentation qmp: Ensure consistent granularity type qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove block: Introduce bdrv_dirty_bitmap_granularity() hbitmap: cache

[Qemu-block] [PATCH v6 15/21] block: Resize bitmaps on bdrv_truncate

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com --- block.c| 18 ++ include/qemu/hbitmap.h | 10 ++ util/hbitmap.c | 48 3 files changed, 76 insertions(+) diff --git a/block.c b/block.c index 735acff

[Qemu-block] [PATCH v6 08/21] block: Add bitmap disabled status

2015-04-17 Thread John Snow
removed for now until a use case emerges where this state must be revealed to the user. The disabled state WILL be used internally for bitmap migration and bitmap persistence. Signed-off-by: Fam Zheng f...@redhat.com Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com

[Qemu-block] [PATCH v6 17/21] iotests: add invalid input incremental backup tests

2015-04-17 Thread John Snow
Signed-off-by: John Snow js...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Stefan Hajnoczi stefa...@redhat.com --- tests/qemu-iotests/124 | 104 + tests/qemu-iotests/124.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files

  1   2   3   4   5   6   7   8   9   10   >