Test the failure case for incremental backups.
Signed-off-by: John Snow
Reviewed-by: Max Reitz
---
tests/qemu-iotests/124 | 57 ++
tests/qemu-iotests/124.out | 4 ++--
2 files changed, 59 insertions(+), 2 deletions(-)
diff --git a/tests/qemu-iot
I am not prepared to send a v3 on this, primarily because I am still
waffling on whether or not to do the code motion patch that is present
in patch #8 of v2 of this series.
However, for the purposes of testing, reviewers may find it convenient
to have a new version of this series that applies
Signed-off-by: John Snow
Reviewed-by: Max Reitz
---
tests/qemu-iotests/124 | 174 +++--
tests/qemu-iotests/124.out | 4 +-
2 files changed, 172 insertions(+), 6 deletions(-)
diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 85675ec.
A filter is added to allow callers to request very specific
events to be pulled from the event queue, while leaving undesired
events still in the stream.
This allows us to poll for completion data for multiple asynchronous
events in any arbitrary order.
A new timeout context is added to the qmp p
The general approach is to set bits close to the boundaries of
where we are truncating and ensure that everything appears to
have gone OK.
We test growing and shrinking by different amounts:
- Less than the granularity
- Less than the granularity, but across a boundary
- Less than sizeof(unsigned
We often don't need the BlockDriverState for functions
that operate on bitmaps. Remove it.
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
---
block.c | 13 ++---
block/backup.c| 2 +-
block/mirror.c| 26 ++
For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"top" sync mode.
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
---
block.c
Test what happens if you fiddle with the granularity.
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 58 +-
tests/qemu-iotests/124.out | 4 ++--
2 files changed, 49 insertions(+), 13 deletions(-)
diff --git a/tests/
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
---
block.c | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/block.c b/block.c
index ca73a6a..30c8568 100644
--- a/block.c
+++ b/block.c
@@ -60,11 +60,11 @@
* or enabled. A frozen bi
A bitmap successor is an anonymous BdrvDirtyBitmap that is intended to
be created just prior to a sensitive operation (e.g. Incremental Backup)
that can either succeed or fail, but during the course of which we still
want a bitmap tracking writes.
On creating a successor, we "freeze" the parent bi
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
---
tests/qemu-iotests/124 | 104 +
tests/qemu-iotests/124.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 110 insertions(+)
create mode 100644 tests
We add a bitmap merge operation to assist in error cases
where we wish to combine two bitmaps together.
This is algorithmically O(bits) provided HBITMAP_LEVELS remains
constant. For a full bitmap on a 64bit machine:
sum(bits/64^k, k, 0, HBITMAP_LEVELS) ~= 1.01587 * bits
We may be able to improve
Add a status indicating the enabled/disabled state of the bitmap.
A bitmap is by default enabled, but you can lock the bitmap into
a read-only state by setting disabled = true.
A previous version of this patch added a QMP interface for changing
the state of the bitmap, but it has since been remove
Add bdrv_clear_dirty_bitmap and a matching QMP command,
qmp_block_dirty_bitmap_clear that enables a user to reset
the bitmap attached to a drive.
This allows us to reset a bitmap in the event of a full
drive backup.
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
R
Signed-off-by: John Snow
---
block.c| 18 ++
include/qemu/hbitmap.h | 10 ++
util/hbitmap.c | 48
3 files changed, 76 insertions(+)
diff --git a/block.c b/block.c
index 735acff..b29aafe 100644
--- a/
Add the "frozen" status booleans, to inform clients
when a bitmap is occupied doing a task.
Signed-off-by: Fam Zheng
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Eric Blake
---
block.c | 1 +
qapi/block-core.json | 5 -
2 files ch
From: Fam Zheng
This field will be set for user created dirty bitmap. Also pass in an
error pointer to bdrv_create_dirty_bitmap, so when a name is already
taken on this BDS, it can report an error message. This is not global
check, two BDSes can have dirty bitmap with a common name.
Implemented
As a convenience: between incremental backups, bitmap migrations
and bitmap persistence we seem to need to recalculate these a lot.
Because the lengths are a little bit-twiddly, let's just solidly
cache them and be done with it.
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
Signed-off-by: John
We treat this field with a variety of different types everywhere
in the code. Now it's just uint32_t.
Signed-off-by: John Snow
Reviewed-by: Eric Blake
Reviewed-by: Max Reitz
Reviewed-by: Stefan Hajnoczi
---
block.c | 11 ++-
block/mirror.c| 4 ++--
inclu
It's spring! The winter snow has thawed and so here is a new
patch series to enter your life and warm your heart.
This patchset enables the in-memory part of the incremental backup
feature, without transactional support.
Support for transactions was separated into a separate series which
is also
The new command pair is added to manage a user created dirty bitmap. The
dirty bitmap's name is mandatory and must be unique for the same device,
but different devices can have bitmaps with the same names.
The granularity is an optional field. If it is not specified, we will
choose a default granu
Signed-off-by: John Snow
---
docs/bitmaps.md | 352
1 file changed, 352 insertions(+)
create mode 100644 docs/bitmaps.md
diff --git a/docs/bitmaps.md b/docs/bitmaps.md
new file mode 100644
index 000..f066b48
--- /dev/null
+++ b/docs/b
This returns the granularity (in bytes) of dirty bitmap,
which matches the QMP interface and the existing query
interface.
Signed-off-by: John Snow
Reviewed-by: Max Reitz
Reviewed-by: Eric Blake
Reviewed-by: Stefan Hajnoczi
---
block.c | 8 ++--
include/block/block.h | 1 +
On 04/17/2015 11:41 AM, Max Reitz wrote:
On 27.03.2015 20:19, John Snow wrote:
The goal here is to add a new method to transactions that allows
developers to specify a callback that will get invoked only once
all jobs spawned by a transaction are completed, allowing developers
the chance to pe
On 04/08/2015 04:19 PM, John Snow wrote:
> For "dirty-bitmap" sync mode, the block job will iterate through the
> given dirty bitmap to decide if a sector needs backup (backup all the
> dirty clusters and skip clean ones), just as allocation conditions of
> "top" sync mode.
>
> Signed-off-by: Fam
On 04/17/2015 06:51 PM, Eric Blake wrote:
On 04/08/2015 04:19 PM, John Snow wrote:
For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"t
On 04/08/2015 04:19 PM, John Snow wrote:
> Add bdrv_clear_dirty_bitmap and a matching QMP command,
> qmp_block_dirty_bitmap_clear that enables a user to reset
> the bitmap attached to a drive.
>
> This allows us to reset a bitmap in the event of a full
> drive backup.
>
> Signed-off-by: John Snow
On 04/17/2015 05:02 PM, John Snow wrote:
>>> { 'type': 'DriveBackup',
>>> 'data': { 'device': 'str', 'target': 'str', '*format': 'str',
>>> 'sync': 'MirrorSyncMode', '*mode': 'NewImageMode',
>>> -'*speed': 'int',
>>> +'*speed': 'int', '*bitmap': 'str',
>
On 04/17/2015 06:43 PM, Eric Blake wrote:
On 04/08/2015 04:19 PM, John Snow wrote:
A bitmap successor is an anonymous BdrvDirtyBitmap that is intended to
be created just prior to a sensitive operation (e.g. Incremental Backup)
that can either succeed or fail, but during the course of which we
On 04/08/2015 04:19 PM, John Snow wrote:
> Add the "frozen" status booleans, to inform clients
> when a bitmap is occupied doing a task.
>
> Signed-off-by: Fam Zheng
> Signed-off-by: John Snow
> Reviewed-by: Max Reitz
> Reviewed-by: Stefan Hajnoczi
> ---
> block.c | 1 +
> qapi/b
On 04/08/2015 04:19 PM, John Snow wrote:
> A bitmap successor is an anonymous BdrvDirtyBitmap that is intended to
> be created just prior to a sensitive operation (e.g. Incremental Backup)
> that can either succeed or fail, but during the course of which we still
> want a bitmap tracking writes.
>
On 04/17/2015 11:23 AM, Eric Blake wrote:
On 04/08/2015 04:19 PM, John Snow wrote:
We add a bitmap merge operation to assist in error cases
where we wish to combine two bitmaps together.
This is algorithmically O(bits) provided HBITMAP_LEVELS remains
constant. For a full bitmap on a 64bit mac
On 04/17/2015 09:33 AM, Max Reitz wrote:
On 09.04.2015 00:20, John Snow wrote:
A filter is added to allow callers to request very specific
events to be pulled from the event queue, while leaving undesired
events still in the stream.
This allows to poll for completion data for multiple asynchr
On 27.03.2015 20:20, John Snow wrote:
Use a transaction to request an incremental backup across two drives.
Coerce one of the jobs to fail, and then re-run the transaction.
Verify that no bitmap data was lost due to the partial transaction
failure.
Signed-off-by: John Snow
---
tests/qemu-iot
On 04/17/2015 10:33 AM, Max Reitz wrote:
On 09.04.2015 00:20, John Snow wrote:
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 174
+++--
tests/qemu-iotests/124.out | 4 +-
2 files changed, 172 insertions(+), 6 deletions(-)
I was happi
On 27.03.2015 20:20, John Snow wrote:
This patch actually implements the transactional callback system
for the drive_backup transaction.
This line is probably not intended to be indented (I always wanted to
make that pun).
(1) We manually pick up a reference to the bitmap if present to
On 04/17/2015 09:25 AM, Max Reitz wrote:
On 09.04.2015 00:19, John Snow wrote:
Signed-off-by: John Snow
---
block.c| 18 ++
include/qemu/hbitmap.h | 10 ++
util/hbitmap.c | 48
3 files change
On 04/17/2015 10:01 AM, Max Reitz wrote:
> On 27.03.2015 20:20, John Snow wrote:
>> In general, since transactions may reference QMP function helpers,
>> it would be nice for them to sit beneath them.
>>
>> This will avoid the need for forward declaring any QMP interfaces,
>> which would be aggrava
On 04/17/2015 12:01 PM, Max Reitz wrote:
On 27.03.2015 20:20, John Snow wrote:
In general, since transactions may reference QMP function helpers,
it would be nice for them to sit beneath them.
This will avoid the need for forward declaring any QMP interfaces,
which would be aggravating to upd
On 04/17/2015 09:50 AM, John Snow wrote:
>
>
> On 04/17/2015 11:06 AM, Eric Blake wrote:
>> On 04/08/2015 04:19 PM, John Snow wrote:
>>> Reviewed-by: Max Reitz
>>> Signed-off-by: John Snow
>>> ---
>>> docs/bitmaps.md | 311
>>>
>>> 1 f
On 04/17/2015 09:17 AM, Max Reitz wrote:
On 09.04.2015 00:19, John Snow wrote:
For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"top"
On 27.03.2015 20:20, John Snow wrote:
We'd like to be able to specify the callback given to backup_start
manually in the case of transactions, so split apart qmp_drive_backup
into an implementation and a wrapper.
Switch drive_backup_prepare to use the new wrapper, but don't overload
the callback
On 27.03.2015 20:20, John Snow wrote:
In general, since transactions may reference QMP function helpers,
it would be nice for them to sit beneath them.
This will avoid the need for forward declaring any QMP interfaces,
which would be aggravating to update in so many places.
Signed-off-by: John
On 04/17/2015 11:06 AM, Eric Blake wrote:
On 04/08/2015 04:19 PM, John Snow wrote:
Reviewed-by: Max Reitz
Signed-off-by: John Snow
---
docs/bitmaps.md | 311
1 file changed, 311 insertions(+)
create mode 100644 docs/bitmaps.md
di
On 27.03.2015 20:20, John Snow wrote:
Allow bitmap successors to carry reference counts.
We can in a later patch use this ability to clean up the dirty bitmap
according to both the individual job's success and the success of all
jobs in the transaction group.
The code for cleaning up a bitmap i
On 27.03.2015 20:19, John Snow wrote:
The goal here is to add a new method to transactions that allows
developers to specify a callback that will get invoked only once
all jobs spawned by a transaction are completed, allowing developers
the chance to perform actions conditionally pending complete
On 27.03.2015 20:20, John Snow wrote:
If we want to get at the job after the life of the job,
we'll need a refcount for this object.
This may occur for example if we wish to inspect the actions
taken by a particular job after a transactional group of jobs
runs, and further actions are required.
On 04/08/2015 04:19 PM, John Snow wrote:
> We add a bitmap merge operation to assist in error cases
> where we wish to combine two bitmaps together.
>
> This is algorithmically O(bits) provided HBITMAP_LEVELS remains
> constant. For a full bitmap on a 64bit machine:
> sum(bits/64^k, k, 0, HBITMAP_
On 04/08/2015 04:19 PM, John Snow wrote:
> As a convenience: between incremental backups, bitmap migrations
> and bitmap persistence we seem to need to recalculate these a lot.
>
> Because the lengths are a little bit-twiddly, let's just solidly
> cache them and be done with it.
>
> Reviewed-by:
On 27.03.2015 20:19, John Snow wrote:
Now that the structure formerly known as BlkTransactionState has been
renamed to something sensible (BlkActionState), re-introduce an actual
BlkTransactionState that actually manages state for the entire Transaction.
In the process, convert the old QSIMPLEQ
On 04/08/2015 04:19 PM, John Snow wrote:
> Reviewed-by: Max Reitz
> Signed-off-by: John Snow
> ---
> docs/bitmaps.md | 311
>
> 1 file changed, 311 insertions(+)
> create mode 100644 docs/bitmaps.md
>
> diff --git a/docs/bitmaps.md b/do
On 04/08/2015 04:19 PM, John Snow wrote:
> The new command pair is added to manage a user created dirty bitmap. The
> dirty bitmap's name is mandatory and must be unique for the same device,
> but different devices can have bitmaps with the same names.
>
> The granularity is an optional field. If
On 27.03.2015 20:19, John Snow wrote:
These structures are misnomers, somewhat.
(1) BlockTransactionState is not state for a transaction,
but is rather state for a single transaction action.
Rename it "BlkActionState" to be more accurate.
(2) The BdrvActionOps describes operations for
On 03/27/2015 01:19 PM, John Snow wrote:
> This adds two qmp commands to transactions.
>
> block-dirty-bitmap-add allows you to create a bitmap simultaneously
> alongside a new full backup to accomplish a clean synchronization
> point.
>
> block-dirty-bitmap-clear allows you to reset a bitmap bac
On 27.03.2015 20:19, John Snow wrote:
Test simple usage cases for using transactions to create
and synchronize incremental backups.
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 51 ++
tests/qemu-iotests/124.out | 4 ++--
2 files chan
On 09.04.2015 00:20, John Snow wrote:
Test what happens if you fiddle with the granularity.
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 58 +-
tests/qemu-iotests/124.out | 4 ++--
2 files changed, 49 insertions(+), 13 deletions(-)
On 09.04.2015 00:20, John Snow wrote:
Test the failure case for incremental backups.
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 57 ++
tests/qemu-iotests/124.out | 4 ++--
2 files changed, 59 insertions(+), 2 deletions(-)
Reviewe
On 27.03.2015 20:19, John Snow wrote:
This adds two qmp commands to transactions.
block-dirty-bitmap-add allows you to create a bitmap simultaneously
alongside a new full backup to accomplish a clean synchronization
point.
block-dirty-bitmap-clear allows you to reset a bitmap back to as-if
it w
On 09.04.2015 00:20, John Snow wrote:
Signed-off-by: John Snow
---
tests/qemu-iotests/124 | 174 +++--
tests/qemu-iotests/124.out | 4 +-
2 files changed, 172 insertions(+), 6 deletions(-)
I was happier with the previous simpler approach, but I
On 09.04.2015 00:20, John Snow wrote:
A filter is added to allow callers to request very specific
events to be pulled from the event queue, while leaving undesired
events still in the stream.
This allows to poll for completion data for multiple asynchronous
events in any arbitrary order.
A new
On 09.04.2015 00:19, John Snow wrote:
Signed-off-by: John Snow
---
block.c| 18 ++
include/qemu/hbitmap.h | 10 ++
util/hbitmap.c | 48
3 files changed, 76 insertions(+)
diff --git a/block.c b/
On 09.04.2015 00:19, John Snow wrote:
For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"top" sync mode.
Signed-off-by: Fam Zheng
Signed
On 16.04.2015 16:30, Alberto Garcia wrote:
On Wed 15 Apr 2015 06:09:18 PM CEST, Max Reitz wrote:
+orig_bs_flags = bdrv_get_flags(bs);
+if (!(orig_bs_flags & BDRV_O_RDWR)) {
I feel like we don't want to do this if we're not streaming to an
intermediate layer but to the top layer (becau
On Fri 17 Apr 2015 01:52:43 PM CEST, Alberto Garcia wrote:
> Anyone calling bdrv_block_device_info() directly will get a null image
> field. As a consequence of this, the HMP command 'info block -n -v'
> crashes QEMU.
>
> This patch moves the code that fills in that field from
> bdrv_query_info()
The image field in BlockDeviceInfo is supposed to contain an ImageInfo
object. However that is being filled in by bdrv_query_info(), not by
bdrv_block_device_info(), which is where BlockDeviceInfo is actually
created.
Anyone calling bdrv_block_device_info() directly will get a null image
field. As
Am 16.04.2015 um 14:34 hat Alberto Garcia geschrieben:
> On Thu 16 Apr 2015 02:27:39 PM CEST, Eric Blake wrote:
>
> +orig_bs_flags = bdrv_get_flags(bs);
> +if (!(orig_bs_flags & BDRV_O_RDWR)) {
> >>>
> >>> I feel like we don't want to do this if we're not streaming to an
> >>> in
Am 16.04.2015 um 15:20 schrieb Paolo Bonzini:
>
> On 16/04/2015 14:58, Peter Lieven wrote:
>>> On 16/04/2015 14:18, Peter Lieven wrote:
We need this to support SCSI_STATUS_TASK_SET_FULL.
>>> Any reason apart from the missing constant?
>> No, but I wanted to avoid starting checking for constant
Am 16.04.2015 um 15:17 schrieb Paolo Bonzini:
>
> On 16/04/2015 15:02, Peter Lieven wrote:
>>> Also, I think it is iscsi_co_generic_cb that should set
>>> force_next_flush, so that it is only set on failure. Not really for the
>>> optimization value, but because it's clearer.
>> I don't get what y
68 matches
Mail list logo
