On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The qemu-img dd/convert commands will create a image file and
> then try to open it. Historically it has been possible to open
> new files without passing any options. With encrypted files
> though, the *key-secret options are mandatory, so we need
On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The '--image-opts' flags indicates whether the source filename
> includes options. The target filename has to remain in the
> plain filename format though, since it needs to be passed to
> bdrv_create(). When using --skip-create though, it would be
On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The -o arg to the convert command allows specification of format/protocol
> options for the newly created image. This adds a -o arg to the dd command
> to get feature parity.
>
> Signed-off-by: Daniel P. Berrange
> ---
>
On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The -n arg to the convert command allows use of a pre-existing image,
> rather than creating a new image. This adds equivalent functionality
> to the dd command using the 'conv' arg. If 'conv=nocreat' is used,
> then it will assume the image already
On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The --image-opts flag can only be used to affect the parsing
> of the source image. The target image has to be specified in
> the traditional style regardless, since it needs to be passed
> to the bdrv_create() API which does not support the new
On 03.02.2017 13:02, Daniel P. Berrange wrote:
> The qemu-img dd command added --image-opts support, but missed
> the corresponding --object support. This prevented passing
> secrets (eg auth passwords) needed by certain disk images.
>
> Reviewed-by: Eric Blake
>
I like both JSON and dot syntax. But I like them differently in
different places.
I love JSON when it's in some file where I can turn out syntax
highlighting and let my $EDITOR match brackets and braces.
I hate JSON when it's on the command line. You have to escape, you get
strings in strings,
On 03.02.2017 08:50, Markus Armbruster wrote:
> "Dr. David Alan Gilbert" writes:
>
>> * Markus Armbruster (arm...@redhat.com) wrote:
>>> = Introduction =
>>>
>>
>>
>>
>>> = Structured option argument syntax =
>>>
>>> == JSON ==
>>>
>>> The obvious way to provide the
When using file system that does not support fallocate() (e.g. NFS <
4.2), truncating the file only when preallocation=OFF speeds up creating
raw file.
Here is example run, tested on Fedora 24 machine, creating raw file on
NFS version 3 server.
$ time ./qemu-img-master create -f raw -o
On 03.02.2017 12:08, Vladimir Sementsov-Ogievskiy wrote:
> 03.02.2017 13:10, Vladimir Sementsov-Ogievskiy wrote:
>> 02.02.2017 02:12, Max Reitz wrote:
>>> On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote:
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one,
On 03.02.2017 11:10, Vladimir Sementsov-Ogievskiy wrote:
> 02.02.2017 02:12, Max Reitz wrote:
>> On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote:
>>> The test starts two vms (vm_a, vm_b), create dirty bitmap in
>>> the first one, do several writes to corresponding device and
>>> then
On 02.02.2017 09:52, Alberto Garcia wrote:
> On Wed 01 Feb 2017 11:16:38 PM CET, Max Reitz wrote:
>
>> Thanks, applied to my block tree, with
>> %s/INT_MAX/BDRV_REQUEST_MAX_BYTES/g:
>
> I think you can use %d to print BDRV_REQUEST_MAX_BYTES, after all the
> definition
On 02.02.2017 11:41, Vladimir Sementsov-Ogievskiy wrote:
> 01.02.2017 02:20, Max Reitz wrote:
>> On 23.01.2017 13:10, Vladimir Sementsov-Ogievskiy wrote:
>>> Realize .bdrv_remove_persistent_dirty_bitmap interface.
>>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy
>>>
On 02.02.2017 08:32, Markus Armbruster wrote:
> Max Reitz writes:
>
>> On 01.02.2017 13:28, Daniel P. Berrange wrote:
>>> On Wed, Feb 01, 2017 at 01:23:54PM +0100, Max Reitz wrote:
On 01.02.2017 13:16, Daniel P. Berrange wrote:
> On Wed, Feb 01, 2017 at 01:13:39PM
On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote:
> There's also the -drive file=json:... syntax. It's a bad fit for
> QemuOpts, because QemuOpts and JSON fight for the comma. I'd show you
> if I could get it to work.
I think this refers to the json: syntax for qemu URIs?
* Markus Armbruster (arm...@redhat.com) wrote:
> "Dr. David Alan Gilbert" writes:
>
> > * Markus Armbruster (arm...@redhat.com) wrote:
> >> = Introduction =
> >>
> >
> >
> >
> >> = Structured option argument syntax =
> >>
> >> == JSON ==
> >>
> >> The obvious way to
Minimal realization: only one extent in server answer is supported.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/nbd.h | 3 ++
nbd/nbd-internal.h | 1 +
nbd/server.c| 80 -
3 files changed,
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
tests/qemu-iotests/180 | 133 +
tests/qemu-iotests/180.out | 5 ++
tests/qemu-iotests/group | 1 +
3 files changed, 139 insertions(+)
create mode 100755
Only one meta context type is defined: qemu-bitmap:.
Maximum one query is allowed for NBD_OPT_{SET,LIST}_META_CONTEXT,
NBD_REP_ERR_TOO_BIG is returned otherwise.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
include/block/nbd.h | 15 ++
nbd/nbd-internal.h | 6 +
Return 0 on success to simplify success checking.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
nbd/client.c | 35 +++
1 file changed, 19 insertions(+), 16 deletions(-)
diff --git a/nbd/client.c b/nbd/client.c
index
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/nbd-client.c | 146 ++-
block/nbd-client.h | 6 +++
block/nbd.c | 9 +++-
include/block/nbd.h | 6 ++-
nbd/client.c | 103
Rename functions appropriately and also make a separate copy of NBDReply
- NBDSimpleReply, to replace NBDReply for the server. NBDReply itself
will be upgraded in future patches to handle both simple and structured
replies in the client.
Signed-off-by: Vladimir Sementsov-Ogievskiy
For loading dirty bitmap from nbd server. Or for underlying storages for
other formats.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
blockdev.c | 28
qapi/block-core.json | 14 ++
2 files changed, 42 insertions(+)
Minimal implementation: always send DF flag, to not deal with fragmented
replies.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/nbd-client.c | 47 +++
block/nbd-client.h | 2 +
include/block/nbd.h | 15 +++--
nbd/client.c| 170
Split out nbd_receive_simple_option to be reused for structured reply
option.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
nbd/client.c | 54 +++---
nbd/nbd-internal.h | 14 ++
2 files changed, 49
Minimal implementation of structured read: one data chunk + finishing
none chunk. No segmentation.
Minimal structured error implementation: no text message.
Support DF flag, but just ignore it, as there is no segmentation any
way.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Realize bdrv_dirty_bitmap_load interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/nbd.c | 32
1 file changed, 32 insertions(+)
diff --git a/block/nbd.c b/block/nbd.c
index 63bc3f04d0..b2b6fd1cf9 100644
--- a/block/nbd.c
The function searches for next zero bit.
Also add interface for BdrvDirtyBitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/dirty-bitmap.c | 5 +
include/block/dirty-bitmap.h | 2 ++
include/qemu/hbitmap.h | 8
util/hbitmap.c
Comparison symbol is misused. It may lead to memory corruption.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
nbd/client.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/nbd/client.c b/nbd/client.c
index 6caf6bda6d..351731bc63 100644
---
Minimal realization: only first extent from the answer is used.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/nbd-client.c | 41 -
block/nbd-client.h | 5 +
block/nbd.c | 3 +++
include/block/nbd.h | 2 +-
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/dirty-bitmap.c | 7 +++
include/block/dirty-bitmap.h | 3 +++
2 files changed, 10 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/dirty-bitmap.c | 53
include/block/block_int.h| 4
include/block/dirty-bitmap.h | 3 +++
3 files changed, 60 insertions(+)
diff --git
Hi all!
We really need exporting dirty bitmaps feature as well as remote
get_block_status for nbd devices. So, here is minimalistic and restricted
realization of 'structured reply' and 'block status' nbd protocol extension
(as second is developed over the first the combined spec may be found
To be consistent when NBD_STRUCTURED_REPLY_MAGIC will be introduced.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
nbd/client.c | 4 ++--
nbd/nbd-internal.h | 2 +-
nbd/server.c | 4 ++--
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/dirty-bitmap.c | 5 +
blockdev.c | 29 +
include/block/dirty-bitmap.h | 2 ++
include/qemu/hbitmap.h | 8
qapi/block-core.json | 27
Fam Zheng writes:
> On Fri, 02/03 14:35, Markus Armbruster wrote:
>> > I guess the painpoint is "okay, what the heck does the machine support
>> > then?" -
>>
>> At the place where we can reliably detect orphaned drives regardless of
>> what the machine initialization code
On Fri, 02/03 14:35, Markus Armbruster wrote:
> > I guess the painpoint is "okay, what the heck does the machine support
> > then?" -
>
> At the place where we can reliably detect orphaned drives regardless of
> what the machine initialization code does, we have no idea.
>
> > that "3 > 2" is
Peter Krempa writes:
> On Thu, Feb 02, 2017 at 20:42:33 +0100, Markus Armbruster wrote:
>
> [...]
>
>> === Comparison ===
>>
>> In my opinion, dotted keys are weird and ugly, but at least they don't
>> add to the quoting mess. Structured values look better, except when
>>
Fam Zheng writes:
> On Mon, 01/30 09:10, Markus Armbruster wrote:
>> John Snow writes:
>>
>> > On 01/27/2017 11:04 AM, Markus Armbruster wrote:
>> >> John Snow writes:
>> >>
>> >>> On 01/27/2017 06:51 AM, Markus Armbruster wrote:
>>
On Thu, Feb 02, 2017 at 20:42:33 +0100, Markus Armbruster wrote:
[...]
> === Comparison ===
>
> In my opinion, dotted keys are weird and ugly, but at least they don't
> add to the quoting mess. Structured values look better, except when
> they do add to the quoting mess.
From libvirt's point
The '--image-opts' flags indicates whether the source filename
includes options. The target filename has to remain in the
plain filename format though, since it needs to be passed to
bdrv_create(). When using --skip-create though, it would be
possible to use image-opts syntax. This adds
The qemu-img dd command added --image-opts support, but missed
the corresponding --object support. This prevented passing
secrets (eg auth passwords) needed by certain disk images.
Reviewed-by: Eric Blake
Signed-off-by: Daniel P. Berrange
---
qemu-img.c
The -o arg to the convert command allows specification of format/protocol
options for the newly created image. This adds a -o arg to the dd command
to get feature parity.
Signed-off-by: Daniel P. Berrange
---
qemu-img-cmds.hx | 2 +-
qemu-img.c | 32
The qemu-img dd/convert commands will create a image file and
then try to open it. Historically it has been possible to open
new files without passing any options. With encrypted files
though, the *key-secret options are mandatory, so we need to
provide those options when opening the newly created
The -n arg to the convert command allows use of a pre-existing image,
rather than creating a new image. This adds equivalent functionality
to the dd command using the 'conv' arg. If 'conv=nocreat' is used,
then it will assume the image already exists. The existing image
will be truncated to match
The --image-opts flag can only be used to affect the parsing
of the source image. The target image has to be specified in
the traditional style regardless, since it needs to be passed
to the bdrv_create() API which does not support the new style
opts.
Signed-off-by: Daniel P. Berrange
"Daniel P. Berrange" writes:
> On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote:
>> === Dotted keys ===
>>
>> One sufficiently powerful syntax extension already exists: the dotted
>> key convention. It's syntactically unambiguous only when none of the
>>
01.02.2017 14:06, Vladimir Sementsov-Ogievskiy wrote:
24.01.2017 22:53, Dr. David Alan Gilbert wrote:
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote:
24.01.2017 12:24, Juan Quintela wrote:
Vladimir Sementsov-Ogievskiy wrote:
Split common postcopy
03.02.2017 13:10, Vladimir Sementsov-Ogievskiy wrote:
02.02.2017 02:12, Max Reitz wrote:
On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote:
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b
On Mon, 01/30 09:10, Markus Armbruster wrote:
> John Snow writes:
>
> > On 01/27/2017 11:04 AM, Markus Armbruster wrote:
> >> John Snow writes:
> >>
> >>> On 01/27/2017 06:51 AM, Markus Armbruster wrote:
> John Snow writes:
>
>
Hi all!
There is a new update of qcow2-bitmap series - v13.
web:
https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v13
git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v13)
v13: Just a fix for style checker.
13: line over 80
14: line over 80
22:
02.02.2017 02:12, Max Reitz wrote:
On 22.11.2016 18:54, Vladimir Sementsov-Ogievskiy wrote:
The test starts two vms (vm_a, vm_b), create dirty bitmap in
the first one, do several writes to corresponding device and
then migrate vm_a to vm_b with dirty bitmaps.
Signed-off-by: Vladimir
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be
needed in future, to save this flag back to Qcow2 for persistent
bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/dirty-bitmap.c | 16
On Thu, Feb 02, 2017 at 08:42:33PM +0100, Markus Armbruster wrote:
> === Dotted keys ===
>
> One sufficiently powerful syntax extension already exists: the dotted
> key convention. It's syntactically unambiguous only when none of the
> KEYs involved contains '.' To adopt it across the board,
Optional. Default is false.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Signed-off-by: Denis V. Lunev
Reviewed-by: Max Reitz
---
blockdev.c | 18 --
qapi/block-core.json | 6 +-
2 files changed, 21
Add bitmap extension as specified in docs/specs/qcow2.txt.
For now, just mirror extension header into Qcow2 state and check
constraints.
For now, disable image resize if it has bitmaps. It will be fixed later.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
This is needed for the following patch, which will introduce refcounts
checking for qcow2 bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/qcow2-refcount.c | 53 ++
A bitmap directory entry is sometimes called a 'bitmap header'. This
patch leaves only one name - 'bitmap directory entry'. The name 'bitmap
header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap
header extension' (which is extension of qcow2 header)
Signed-off-by: Vladimir
Interface for removing persistent bitmap from its storage.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/dirty-bitmap.c | 18 ++
include/block/block_int.h| 3 +++
include/block/dirty-bitmap.h | 3 +++
3 files changed, 24
Make dirty iter resistant to resetting bits in corresponding HBitmap.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
Reviewed-by: John Snow
---
include/qemu/hbitmap.h | 26 --
Remove persistent bitmap from the storage on block-dirty-bitmap-remove.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
blockdev.c | 10 ++
1 file changed, 10 insertions(+)
diff --git a/blockdev.c b/blockdev.c
index c41b791289..a365cdf3ed 100644
---
Realize block bitmap storing interface, to allow qcow2 images store
persistent bitmaps.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/qcow2-bitmap.c | 489 +--
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add.
Default is false.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Signed-off-by: Denis V. Lunev
Reviewed-by: Max Reitz
---
blockdev.c | 18
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
tests/qemu-iotests/165 | 89 ++
tests/qemu-iotests/165.out | 5 +++
tests/qemu-iotests/group | 1 +
3 files changed, 95
Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img
check implementation.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/qcow2-bitmap.c | 76 ++
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/dirty-bitmap.c | 7 +++
include/block/dirty-bitmap.h | 3 +++
2 files changed, 10 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index
Realize .bdrv_remove_persistent_dirty_bitmap interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy
---
block/qcow2-bitmap.c | 41 +
block/qcow2.c| 1 +
block/qcow2.h| 3 +++
3 files changed, 45 insertions(+)
Realize .bdrv_can_store_new_dirty_bitmap interface.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/qcow2-bitmap.c | 40
block/qcow2.c| 1 +
block/qcow2.h| 4
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
---
docs/specs/qcow2.txt | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
index 80cdfd0e91..dda53dd2a3 100644
---
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They
are loaded when the image is opened and become BdrvDirtyBitmaps for the
corresponding drive.
Extra data in bitmaps is not supported for now.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by:
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for
qcow2 bitmap loading, to handle unallocated bitmap parts, marked as
all-ones.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Kevin Wolf
Reviewed-by: John Snow
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block/dirty-bitmap.c | 5 +
blockdev.c | 29 +
include/block/dirty-bitmap.h | 2 ++
include/qemu/hbitmap.h | 8
This will be needed to check some restrictions before making bitmap
persistent in qmp-block-dirty-bitmap-add (this functionality will be
added by future patch)
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: Max Reitz
---
block.c
Test that hbitmap iter is resistant to bitmap resetting.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Signed-off-by: Denis V. Lunev
Reviewed-by: Max Reitz
Reviewed-by: John Snow
---
tests/test-hbitmap.c | 19
Auto loading bitmaps are bitmaps stored in the disk image, which should
be loaded when the image is opened and become BdrvDirtyBitmaps for the
corresponding drive.
Signed-off-by: Vladimir Sementsov-Ogievskiy
Reviewed-by: John Snow
Reviewed-by: Max
Make getter signature const-correct. This allows other functions with
const dirty bitmap parameter use bdrv_dirty_bitmap_granularity().
Reviewed-by: Eric Blake
Reviewed-by: John Snow
Reviewed-by: Kevin Wolf
Signed-off-by: Vladimir
76 matches
Mail list logo