Re: [Qemu-block] [Qemu-devel] [PATCH v8 04/14] block/dirty-bitmap: add bdrv_dirty_bitmap_set_frozen

2017-11-17 Thread John Snow
On 11/17/2017 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: > 17.11.2017 20:20, John Snow wrote: >> >> On 11/17/2017 09:46 AM, Vladimir Sementsov-Ogievskiy wrote: >>> 14.11.2017 02:32, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: > Make it possible to set

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-17 Thread John Snow
On 11/17/2017 02:10 PM, Eric Blake wrote: > On 11/17/2017 01:04 PM, Eric Blake wrote: >> The contents of a qcow2 bitmap are rounded up to a size that >> matches the number of bits available for the granularity, but >> that granularity differs for 32-bit hosts (our default 64k >> cluster allows

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 0/4] qmp dirty bitmap API

2017-11-17 Thread John Snow
On 11/17/2017 03:22 AM, Vladimir Sementsov-Ogievskiy wrote: > 17.11.2017 06:10, John Snow wrote: >> >> On 11/16/2017 03:17 AM, Vladimir Sementsov-Ogievskiy wrote: >>> 16.11.2017 00:20, John Snow wrote: On 11/13/2017 11:20 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all. > >

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread John Snow
On 11/17/2017 01:25 PM, Kevin Wolf wrote: > Am 17.11.2017 um 19:15 hat John Snow geschrieben: >> >> >> On 11/17/2017 10:01 AM, Max Reitz wrote: >>> On 2017-11-17 13:30, Kevin Wolf wrote: Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: > Snapshot-switch actually

Re: [Qemu-block] [PATCH v4 for-2.11] QAPI & interop: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Kashyap Chamarthy
On Fri, Nov 17, 2017 at 07:54:44PM +0100, Kashyap Chamarthy wrote: > When you cancel an in-progress 'mirror' job (or "active `block-commit`) When applying, can the maintainer please fix-up the missing closing double-quote in the brackets above? It should be: "active `block-commit`" [...] --

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-17 Thread Eric Blake
On 11/17/2017 01:04 PM, Eric Blake wrote: > The contents of a qcow2 bitmap are rounded up to a size that > matches the number of bits available for the granularity, but > that granularity differs for 32-bit hosts (our default 64k > cluster allows for 2M bitmap coverage per 'long') and 64-bit >

[Qemu-block] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-17 Thread Eric Blake
The contents of a qcow2 bitmap are rounded up to a size that matches the number of bits available for the granularity, but that granularity differs for 32-bit hosts (our default 64k cluster allows for 2M bitmap coverage per 'long') and 64-bit hosts (4M bitmap per 'long'). If the image is a

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Eric Blake
On 11/17/2017 12:46 PM, Eric Blake wrote: >> >> The test fails with -m32 and probably also on Big Endian architectures, >> because the bitmap hash differs. > > Oh my. Thanks for the rapid testing. > >> We could "fix" this by replacing the 2100 by a 2102, so for both bit >> widths rounding is

[Qemu-block] [PATCH v4 for-2.11] QAPI & interop: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Kashyap Chamarthy
When you cancel an in-progress 'mirror' job (or "active `block-commit`) with QMP `block-job-cancel`, it emits the event: BLOCK_JOB_CANCELLED. However, when `block-job-cancel` is issued *after* `drive-mirror` has indicated (via the event BLOCK_JOB_READY) that the source and destination have reached

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Eric Blake
On 11/17/2017 12:17 PM, Max Reitz wrote: > On 2017-11-17 17:47, Eric Blake wrote: >> If an image contains persistent snapshots, we cannot use the >> fast path of bdrv_make_empty() to clear the image during >> qemu-img commit, because that will lose the clusters related >> to the bitmaps. >> >>

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 19:15 hat John Snow geschrieben: > > > On 11/17/2017 10:01 AM, Max Reitz wrote: > > On 2017-11-17 13:30, Kevin Wolf wrote: > >> Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: > >>> Snapshot-switch actually changes active state of disk so it should > >>>

Re: [Qemu-block] [Qemu-devel] [PATCH v8 02/14] block/dirty-bitmap: add locked version of bdrv_release_dirty_bitmap

2017-11-17 Thread John Snow
On 11/17/2017 03:07 AM, Vladimir Sementsov-Ogievskiy wrote: > 11.11.2017 01:52, John Snow wrote: >> >> On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: >>> It is needed to realize bdrv_dirty_bitmap_release_successor in >>> the following patch. >>> >> OK, but... >> >>> Signed-off-by:

[Qemu-block] [PULL 22/25] qcow2: Refuse to get unaligned offsets from cache

2017-11-17 Thread Kevin Wolf
From: Max Reitz Instead of using an assertion, it is better to emit a corruption event here. Checking all offsets for correct alignment can be tedious and it is easily possible to forget to do so. qcow2_cache_do_get() is a function every L2 and refblock access has to go

[Qemu-block] [PULL 25/25] iotests: Make 087 pass without AIO enabled

2017-11-17 Thread Kevin Wolf
From: Max Reitz If AIO has not been enabled in the qemu build that is to be tested, we should skip the "aio=native without O_DIRECT" test instead of failing. Signed-off-by: Max Reitz Message-id: 20171115180732.31753-1-mre...@redhat.com Reviewed-by: Eric

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Max Reitz
On 2017-11-17 17:47, Eric Blake wrote: > If an image contains persistent snapshots, we cannot use the > fast path of bdrv_make_empty() to clear the image during > qemu-img commit, because that will lose the clusters related > to the bitmaps. > > Also leave a comment in qcow2_read_extensions to

[Qemu-block] [PULL 17/25] qcow2: reject unaligned offsets in write compressed

2017-11-17 Thread Kevin Wolf
From: Anton Nefedov Misaligned compressed write is not supported. Signed-off-by: Anton Nefedov Message-id: 1510654613-47868-2-git-send-email-anton.nefe...@virtuozzo.com Reviewed-by: Eric Blake Signed-off-by: Max

[Qemu-block] [PULL 20/25] block: Guard against NULL bs->drv

2017-11-17 Thread Kevin Wolf
From: Max Reitz We currently do not guard everywhere against a NULL bs->drv where we should be doing so. Most of the places fixed here just do not care about that case at all. Some care implicitly, e.g. through a prior function call to bdrv_getlength() which would always

[Qemu-block] [PULL 24/25] block: Make bdrv_next() keep strong references

2017-11-17 Thread Kevin Wolf
From: Max Reitz On one hand, it is a good idea for bdrv_next() to return a strong reference because ideally nearly every pointer should be refcounted. This fixes intermittent failure of iotest 194. On the other, it is absolutely necessary for bdrv_next() itself to keep a

[Qemu-block] [PULL 14/25] iotests: Add test for non-string option reopening

2017-11-17 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Reviewed-by: Eric Blake Message-id: 20171114180128.17076-6-mre...@redhat.com Signed-off-by: Max Reitz --- tests/qemu-iotests/133

[Qemu-block] [PULL 23/25] qcow2: Fix overly broad madvise()

2017-11-17 Thread Kevin Wolf
From: Max Reitz @mem_size and @offset are both size_t, thus subtracting them from one another will just return a big size_t if mem_size < offset -- even more obvious here because the result is stored in another size_t. Checking that result to be positive is therefore not

[Qemu-block] [PULL 21/25] qcow2: Add bounds check to get_refblock_offset()

2017-11-17 Thread Kevin Wolf
From: Max Reitz Reported-by: R. Nageswara Sastry Buglink: https://bugs.launchpad.net/qemu/+bug/1728661 Signed-off-by: Max Reitz Message-id: 20171110203111.7666-5-mre...@redhat.com Reviewed-by: Eric Blake

[Qemu-block] [PULL 18/25] qcow2: check_errors are fatal

2017-11-17 Thread Kevin Wolf
From: Max Reitz When trying to repair a dirty image, qcow2_check() may apparently succeed (no really fatal error occurred that would prevent the check from continuing), but if check_errors in the result object is non-zero, we cannot trust the image to be usable. Reported-by:

[Qemu-block] [PULL 19/25] qcow2: Unaligned zero cluster in handle_alloc()

2017-11-17 Thread Kevin Wolf
From: Max Reitz We should check whether the cluster offset we are about to use is actually valid; that is, whether it is aligned to cluster boundaries. Reported-by: R. Nageswara Sastry Buglink: https://bugs.launchpad.net/qemu/+bug/1728643 Buglink:

[Qemu-block] [PULL 15/25] tests: Add check-qobject for equality tests

2017-11-17 Thread Kevin Wolf
From: Max Reitz Add a new test file (check-qobject.c) for unit tests that concern QObjects as a whole. Its only purpose for now is to test the qobject_is_equal() function. Signed-off-by: Max Reitz Message-id: 20171114180128.17076-7-mre...@redhat.com

[Qemu-block] [PULL 10/25] qapi/qnull: Add own header

2017-11-17 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Reviewed-by: Markus Armbruster Message-id: 20171114180128.17076-2-mre...@redhat.com Signed-off-by: Max

[Qemu-block] [PULL 13/25] block: qobject_is_equal() in bdrv_reopen_prepare()

2017-11-17 Thread Kevin Wolf
From: Max Reitz Currently, bdrv_reopen_prepare() assumes that all BDS options are strings. However, this is not the case if the BDS has been created through the json: pseudo-protocol or blockdev-add. Note that the user-invokable reopen command is an HMP command, so you can

[Qemu-block] [PULL 12/25] qapi: Add qobject_is_equal()

2017-11-17 Thread Kevin Wolf
From: Max Reitz This generic function (along with its implementations for different types) determines whether two QObjects are equal. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia

[Qemu-block] [PULL 09/25] qcow2: fix image corruption on commit with persistent bitmap

2017-11-17 Thread Kevin Wolf
From: Eric Blake If an image contains persistent bitmaps, we cannot use the fast path of bdrv_make_empty() to clear the image during qemu-img commit, because that will lose the clusters related to the bitmaps. Also leave a comment in qcow2_read_extensions to remind future

[Qemu-block] [PULL 06/25] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Kevin Wolf
From: "Daniel P. Berrange" After committing the qcow2 image contents into the base image, qemu-img will call bdrv_make_empty to drop the payload in the layered image. When this is done for qcow2 images, it blows away the LUKS encryption header, making the resulting image

[Qemu-block] [PULL 16/25] iotests: Add test for failing qemu-img commit

2017-11-17 Thread Kevin Wolf
From: Max Reitz Signed-off-by: Max Reitz Message-id: 20170616135847.17726-1-mre...@redhat.com Reviewed-by: Eric Blake Signed-off-by: Max Reitz --- tests/qemu-iotests/020 | 27 +++

[Qemu-block] [PULL 11/25] qapi/qlist: Add qlist_append_null() macro

2017-11-17 Thread Kevin Wolf
From: Max Reitz Besides the macro itself, this patch also adds a corresponding Coccinelle rule. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Message-id:

[Qemu-block] [PULL 07/25] block: Fix permissions in image activation

2017-11-17 Thread Kevin Wolf
Inactive images generally request less permissions for their image files than they would if they were active (in particular, write permissions). Activating the image involves extending the permissions, therefore. drv->bdrv_invalidate_cache() can already require write access to the image file, so

[Qemu-block] [PULL 03/25] block: Fix error path in bdrv_backing_update_filename()

2017-11-17 Thread Kevin Wolf
error_setg_errno() takes a positive errno code. Spotted by Coverity (CID 1381628). Signed-off-by: Kevin Wolf Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi --- block.c | 2 +- 1 file

[Qemu-block] [PULL 00/25] Block layer patches for 2.11.0-rc2

2017-11-17 Thread Kevin Wolf
The following changes since commit fec035a53fa15c4c8c4e62bfef56a35df4161e38: Merge remote-tracking branch 'remotes/kraxel/tags/ui-20171117-pull-request' into staging (2017-11-17 10:18:41 +) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream

[Qemu-block] [PULL 04/25] qcow2: don't permit changing encryption parameters

2017-11-17 Thread Kevin Wolf
From: "Daniel P. Berrange" Currently if trying to change encryption parameters on a qcow2 image, qemu-img will abort. We already explicitly check for attempt to change encrypt.format but missed other parameters like encrypt.key-secret. Rather than list each parameter, just

[Qemu-block] [PULL 02/25] qemu-iotests: Use -nographic in 182

2017-11-17 Thread Kevin Wolf
This avoids that random UI frontend error messages end up in the output. In particular, we were seeing this line in CI error logs: +Unable to init server: Could not connect: Connection refused Signed-off-by: Kevin Wolf Reviewed-by: Kashyap Chamarthy

[Qemu-block] [PULL 08/25] iotests: test clearing unknown autoclear_features by qcow2

2017-11-17 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy Test clearing unknown autoclear_features by qcow2 on incoming migration. [ kwolf: Fixed wait for destination VM startup ] Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 01/25] replication: Fix replication open fail

2017-11-17 Thread Kevin Wolf
From: Wang Guang replication_child_perm request write permissions for all child which will lead bdrv_check_perm fail. replication_child_perm() should request write permissions only if it is writable itself. Signed-off-by: Wang Guang

[Qemu-block] [PULL 05/25] block: Deprecate bdrv_set_read_only() and users

2017-11-17 Thread Kevin Wolf
bdrv_set_read_only() is used by some block drivers to override the read-only option given by the user. This is not how read-only images generally work in QEMU: Instead of second guessing what the user really meant (which currently includes making an image read-only even if the user didn't only use

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread John Snow
On 11/17/2017 10:01 AM, Max Reitz wrote: > On 2017-11-17 13:30, Kevin Wolf wrote: >> Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: >>> Snapshot-switch actually changes active state of disk so it should >>> reflect on dirty bitmaps. Otherwise next incremental backup using

Re: [Qemu-block] [Qemu-devel] [PATCH 09/10] qemu-iotests: remove unused "here" variable

2017-11-17 Thread Eric Blake
On 11/16/2017 11:38 AM, Cleber Rosa wrote: > Another legacy variable that did not convince me it has any > purpose whatsoever. > > Signed-off-by: Cleber Rosa > --- > +++ b/tests/qemu-iotests/001 > @@ -23,7 +23,6 @@ > seq=`basename $0` > echo "QA output created by $seq" > >

Re: [Qemu-block] [Qemu-devel] [PATCH 06/10] qemu-iotests: turn owner variable into a comment

2017-11-17 Thread Eric Blake
On 11/17/2017 07:18 AM, Cleber Rosa wrote: > > > On 11/17/2017 02:19 AM, Paolo Bonzini wrote: >> On 16/11/2017 18:38, Cleber Rosa wrote: >>> This variables has no real use. To avoid pretending it does, while >>> still keeping the information, let's turn it into a comment. >>> >>> The format

Re: [Qemu-block] [Qemu-devel] [PATCH v8 04/14] block/dirty-bitmap: add bdrv_dirty_bitmap_set_frozen

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 20:20, John Snow wrote: On 11/17/2017 09:46 AM, Vladimir Sementsov-Ogievskiy wrote: 14.11.2017 02:32, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: Make it possible to set bitmap 'frozen' without a successor. This is needed to protect the bitmap

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 18:07 hat Vladimir Sementsov-Ogievskiy geschrieben: > 17.11.2017 19:47, Eric Blake wrote: > > If an image contains persistent snapshots, we cannot use the > > bitmaps > > > fast path of bdrv_make_empty() to clear the image during > > qemu-img commit, because that will lose the

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 17:47 hat Eric Blake geschrieben: > If an image contains persistent snapshots, we cannot use the > fast path of bdrv_make_empty() to clear the image during > qemu-img commit, because that will lose the clusters related > to the bitmaps. > > Also leave a comment in

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Eric Blake
On 11/17/2017 11:07 AM, Vladimir Sementsov-Ogievskiy wrote: > 17.11.2017 19:47, Eric Blake wrote: >> If an image contains persistent snapshots, we cannot use the > > bitmaps Oh, right. Maintainer can fix that, if I don't need to respin. > >> fast path of bdrv_make_empty() to clear the image

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 19:47, Eric Blake wrote: If an image contains persistent snapshots, we cannot use the bitmaps fast path of bdrv_make_empty() to clear the image during qemu-img commit, because that will lose the clusters related to the bitmaps. Also leave a comment in qcow2_read_extensions to

Re: [Qemu-block] [PATCH v3 for-2.11] QAPI & interop: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Eric Blake
On 11/17/2017 10:25 AM, Kashyap Chamarthy wrote: > When you cancel an in-progress live block operation with QMP > `block-job-cancel`, it emits the event: BLOCK_JOB_CANCELLED. However, > when `block-job-cancel` is issued after `drive-mirror` has indicated (by > emitting the event BLOCK_JOB_READY)

Re: [Qemu-block] [PATCH] block: Close a BlockDriverState completely even when bs->drv is NULL

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 17:19 hat Alberto Garcia geschrieben: > On Fri 17 Nov 2017 05:14:08 PM CET, Max Reitz wrote: > > On 2017-11-06 15:53, Alberto Garcia wrote: > >> bdrv_close() skips much of its logic when bs->drv is NULL. This is > >> fine when we're closing a BlockDriverState that has just been

Re: [Qemu-block] [PATCH v22 00/30] qcow2: persistent dirty bitmaps

2017-11-17 Thread Eric Blake
On 11/17/2017 10:17 AM, Vladimir Sementsov-Ogievskiy wrote: >> Nothing in this series touched qemu-img.  At the very minimum, it would >> be nice if 'qemu-img info' were to display a summary of all bitmaps in >> the qcow2 file; the QMP type ImageInfoSpecificQCow2 should be updated to >> mention

[Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-17 Thread Eric Blake
If an image contains persistent snapshots, we cannot use the fast path of bdrv_make_empty() to clear the image during qemu-img commit, because that will lose the clusters related to the bitmaps. Also leave a comment in qcow2_read_extensions to remind future feature additions to think about

[Qemu-block] [PATCH v3] QAPI & interop: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Kashyap Chamarthy
When you cancel an in-progress live block operation with QMP `block-job-cancel`, it emits the event: BLOCK_JOB_CANCELLED. However, when `block-job-cancel` is issued after `drive-mirror` has indicated (by emitting the event BLOCK_JOB_READY) that the source and destination remain synchronized:

Re: [Qemu-block] [PATCH] block: Close a BlockDriverState completely even when bs->drv is NULL

2017-11-17 Thread Alberto Garcia
On Fri 17 Nov 2017 05:14:08 PM CET, Max Reitz wrote: > On 2017-11-06 15:53, Alberto Garcia wrote: >> bdrv_close() skips much of its logic when bs->drv is NULL. This is >> fine when we're closing a BlockDriverState that has just been created >> (because e.g the initialization process failed), but

Re: [Qemu-block] [PATCH v22 00/30] qcow2: persistent dirty bitmaps

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 19:04, Eric Blake wrote: Revisiting an old series: On 06/28/2017 07:05 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all! There is a new update of qcow2-bitmap series - v22. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v22 git:

Re: [Qemu-block] segfault in parallel blockjobs (iotest 30)

2017-11-17 Thread Alberto Garcia
On Thu 16 Nov 2017 10:56:58 PM CET, John Snow wrote: I have the impression that one major source of headaches is the fact that the reopen queue contains nodes that don't need to be reopened at all. Ideally this should be detected early on in bdrv_reopen_queue(), so there's no

Re: [Qemu-block] [PATCH] block: Close a BlockDriverState completely even when bs->drv is NULL

2017-11-17 Thread Max Reitz
On 2017-11-06 15:53, Alberto Garcia wrote: > bdrv_close() skips much of its logic when bs->drv is NULL. This is > fine when we're closing a BlockDriverState that has just been created > (because e.g the initialization process failed), but it's not enough > in other cases. > > For example, when a

Re: [Qemu-block] [PATCH v2 for-2.11] block: Make bdrv_next() keep strong references

2017-11-17 Thread Max Reitz
On 2017-11-10 18:25, Max Reitz wrote: > On one hand, it is a good idea for bdrv_next() to return a strong > reference because ideally nearly every pointer should be refcounted. > This fixes intermittent failure of iotest 194. > > On the other, it is absolutely necessary for bdrv_next() itself to

Re: [Qemu-block] [PATCH] iotests: Make 087 pass without AIO enabled

2017-11-17 Thread Max Reitz
On 2017-11-15 19:07, Max Reitz wrote: > If AIO has not been enabled in the qemu build that is to be tested, we > should skip the "aio=native without O_DIRECT" test instead of failing. > > Signed-off-by: Max Reitz > --- > Cleber wanted to fix this in July with his "build

Re: [Qemu-block] [PATCH v22 00/30] qcow2: persistent dirty bitmaps

2017-11-17 Thread Eric Blake
Revisiting an old series: On 06/28/2017 07:05 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > There is a new update of qcow2-bitmap series - v22. > > web: > https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v22 > git:

Re: [Qemu-block] [PATCH for-2.11 2/2] iotests: test clearing unknown autoclear_features by qcow2

2017-11-17 Thread Max Reitz
On 2017-11-17 16:47, Kevin Wolf wrote: > From: Vladimir Sementsov-Ogievskiy > > Test clearing unknown autoclear_features by qcow2 on incoming > migration. > > [ kwolf: Fixed wait for destination VM startup ] > > Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH for-2.11 1/2] block: Fix permissions in image activation

2017-11-17 Thread Max Reitz
On 2017-11-17 16:47, Kevin Wolf wrote: > Inactive images generally request less permissions for their image files > than they would if they were active (in particular, write permissions). > Activating the image involves extending the permissions, therefore. > > drv->bdrv_invalidate_cache() can

[Qemu-block] [PATCH for-2.11 0/2] block: Fix permissions in image activation

2017-11-17 Thread Kevin Wolf
Kevin Wolf (1): block: Fix permissions in image activation Vladimir Sementsov-Ogievskiy (1): iotests: test clearing unknown autoclear_features by qcow2 block.c| 32 +++--- tests/qemu-iotests/196 | 66 ++

[Qemu-block] [PATCH for-2.11 1/2] block: Fix permissions in image activation

2017-11-17 Thread Kevin Wolf
Inactive images generally request less permissions for their image files than they would if they were active (in particular, write permissions). Activating the image involves extending the permissions, therefore. drv->bdrv_invalidate_cache() can already require write access to the image file, so

[Qemu-block] [PATCH for-2.11 2/2] iotests: test clearing unknown autoclear_features by qcow2

2017-11-17 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy Test clearing unknown autoclear_features by qcow2 on incoming migration. [ kwolf: Fixed wait for destination VM startup ] Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 4/4] nbd/server: Fix error reporting for bad requests

2017-11-17 Thread Eric Blake
The NBD spec says an attempt to NBD_CMD_TRIM on a read-only export should fail with EPERM, as a trim has the potential to change disk contents, but we were relying on the block layer to catch that for us, which might not always give the right error (and even if it does, it does not let us pass

[Qemu-block] [PULL 3/4] nbd/client: Don't hard-disconnect on ESHUTDOWN from server

2017-11-17 Thread Eric Blake
The NBD spec says that a server may fail any transmission request with ESHUTDOWN when it is apparent that no further request from the client can be successfully honored. The client is supposed to then initiate a soft shutdown (wait for all remaining in-flight requests to be answered, then send

[Qemu-block] [PULL 1/4] nbd: Don't crash when server reports NBD_CMD_READ failure

2017-11-17 Thread Eric Blake
If a server fails a read, for example with EIO, but the connection is still live, then we would crash trying to print a non-existent error message in nbd_client_co_preadv(). For consistency, also change the error printout in nbd_read_reply_entry(), although that instance does not crash. Bug

[Qemu-block] [PULL 2/4] nbd/client: Use error_prepend() correctly

2017-11-17 Thread Eric Blake
When using error prepend(), it is necessary to end with a space in the format string; otherwise, messages come out incorrectly, such as when connecting to a socket that hangs up immediately: can't open device nbd://localhost:10809/: Failed to read dataUnexpected end-of-file before all bytes were

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Max Reitz
On 2017-11-17 13:30, Kevin Wolf wrote: > Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: >> Snapshot-switch actually changes active state of disk so it should >> reflect on dirty bitmaps. Otherwise next incremental backup using >> these bitmaps will be invalid. >> >>

Re: [Qemu-block] [Qemu-devel] [PATCH v8 04/14] block/dirty-bitmap: add bdrv_dirty_bitmap_set_frozen

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
14.11.2017 02:32, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: Make it possible to set bitmap 'frozen' without a successor. This is needed to protect the bitmap during outgoing bitmap postcopy migration. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v2 for-2.11] nbd/server: Fix error reporting for bad requests

2017-11-17 Thread Eric Blake
On 11/16/2017 02:52 AM, Vladimir Sementsov-Ogievskiy wrote: >>   if (request->type == NBD_CMD_READ || request->type == >> NBD_CMD_WRITE) { >>   if (request->len > NBD_MAX_BUFFER_SIZE) { >>   error_setg(errp, "len (%" PRIu32" ) is larger than max >> len (%u)", > > related

Re: [Qemu-block] [PATCH v2 (for-2.11)] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Daniel P. Berrange
On Fri, Nov 17, 2017 at 08:32:32AM -0600, Eric Blake wrote: > On 11/17/2017 05:29 AM, Daniel P. Berrange wrote: > > After committing the qcow2 image contents into the base image, qemu-img > > will call bdrv_make_empty to drop the payload in the layered image. > > > > When this is done for qcow2

Re: [Qemu-block] [PATCH v2 (for-2.11)] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 15:32 hat Eric Blake geschrieben: > On 11/17/2017 05:29 AM, Daniel P. Berrange wrote: > > After committing the qcow2 image contents into the base image, qemu-img > > will call bdrv_make_empty to drop the payload in the layered image. > > > > When this is done for qcow2 images, it

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] nbd/client: Don't hard-disconnect on ESHUTDOWN from server

2017-11-17 Thread Eric Blake
On 11/13/2017 01:48 PM, Eric Blake wrote: > The NBD spec says that a server may fail any transmission request > with ESHUTDOWN when it is apparent that no further request from > the client can be successfully honored. The client is supposed > to then initiate a soft shutdown (wait for all

Re: [Qemu-block] [Qemu-devel] [PATCH] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Eric Blake
On 11/14/2017 07:52 AM, Max Reitz wrote: >>> Hmm - I wonder if persistent bitmaps are also corrupted in the fast path. >> >> I also wonder if there's anything better we can do to make us safer by >> default, so we default to the slow & safe path, unless we can provide >> we *only* have the subset

Re: [Qemu-block] [PATCH v2 (for-2.11)] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Eric Blake
On 11/17/2017 05:29 AM, Daniel P. Berrange wrote: > After committing the qcow2 image contents into the base image, qemu-img > will call bdrv_make_empty to drop the payload in the layered image. > > When this is done for qcow2 images, it blows away the LUKS encryption > header, making the

Re: [Qemu-block] [ovirt-users] Enabling libgfapi disk access with oVirt 4.2

2017-11-17 Thread Kevin Wolf
Am 15.11.2017 um 23:05 hat Nir Soffer geschrieben: > On Wed, Nov 15, 2017 at 8:58 AM Misak Khachatryan wrote: > > > Hi, > > > > will it be a more clean approach? I can't tolerate full stop of all > > VMs just to enable it, seems too disastrous for real production > >

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Denis V. Lunev
On 11/17/2017 04:42 PM, Kevin Wolf wrote: > Am 17.11.2017 um 13:58 hat Denis V. Lunev geschrieben: >> On 11/17/2017 03:30 PM, Kevin Wolf wrote: >>> Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: Snapshot-switch actually changes active state of disk so it should

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 13:58 hat Denis V. Lunev geschrieben: > On 11/17/2017 03:30 PM, Kevin Wolf wrote: > > Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: > >> Snapshot-switch actually changes active state of disk so it should > >> reflect on dirty bitmaps. Otherwise next

Re: [Qemu-block] [Qemu-devel] [PATCH 06/10] qemu-iotests: turn owner variable into a comment

2017-11-17 Thread Cleber Rosa
On 11/17/2017 02:19 AM, Paolo Bonzini wrote: > On 16/11/2017 18:38, Cleber Rosa wrote: >> This variables has no real use. To avoid pretending it does, while >> still keeping the information, let's turn it into a comment. >> >> The format chosen is the one already being used on tests 149 and

Re: [Qemu-block] [Qemu-devel] [PATCH 01/10] qemu-iotests: make execution of tests agnostic to test type

2017-11-17 Thread Cleber Rosa
On 11/17/2017 02:25 AM, Paolo Bonzini wrote: > On 16/11/2017 18:38, Cleber Rosa wrote: >> check makes a distinction on how it runs Python based tests. The >> current approach is inconsistent because: >> >> 1) a large number of Python tests are already set as executable files >> (eg: 030, 040,

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Denis V. Lunev
On 11/17/2017 03:30 PM, Kevin Wolf wrote: > Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: >> Snapshot-switch actually changes active state of disk so it should >> reflect on dirty bitmaps. Otherwise next incremental backup using >> these bitmaps will be invalid. >> >>

Re: [Qemu-block] [PATCH v2 (for-2.11)] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Kevin Wolf
Am 17.11.2017 um 12:29 hat Daniel P. Berrange geschrieben: > After committing the qcow2 image contents into the base image, qemu-img > will call bdrv_make_empty to drop the payload in the layered image. > > When this is done for qcow2 images, it blows away the LUKS encryption > header, making the

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-17 Thread Kevin Wolf
Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben: > Snapshot-switch actually changes active state of disk so it should > reflect on dirty bitmaps. Otherwise next incremental backup using > these bitmaps will be invalid. > > Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH v2 (for-2.11)] qcow2: fix image corruption after committing qcow2 image into base

2017-11-17 Thread Daniel P. Berrange
After committing the qcow2 image contents into the base image, qemu-img will call bdrv_make_empty to drop the payload in the layered image. When this is done for qcow2 images, it blows away the LUKS encryption header, making the resulting image unusable. There are two codepaths for emptying a

Re: [Qemu-block] [PATCH v2] qapi: block-core: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Kashyap Chamarthy
On Fri, Nov 17, 2017 at 11:49:05AM +0100, Kevin Wolf wrote: > Am 16.11.2017 um 10:14 hat Kashyap Chamarthy geschrieben: > > On Wed, Nov 15, 2017 at 04:56:13PM -0500, John Snow wrote: [...] > > > It's an interesting gotcha that I wasn't really acutely aware of myself, > > > so having it in the

Re: [Qemu-block] [PATCH v2] qapi: block-core: Clarify events emitted by 'block-job-cancel'

2017-11-17 Thread Kevin Wolf
Am 16.11.2017 um 10:14 hat Kashyap Chamarthy geschrieben: > On Wed, Nov 15, 2017 at 04:56:13PM -0500, John Snow wrote: > > > > > > On 11/15/2017 04:54 PM, Kashyap Chamarthy wrote: > > > On Wed, Nov 15, 2017 at 02:15:57PM -0500, John Snow wrote: > > [...] > > > >> is it covered sufficiently in

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 0/4] qmp dirty bitmap API

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 06:10, John Snow wrote: On 11/16/2017 03:17 AM, Vladimir Sementsov-Ogievskiy wrote: 16.11.2017 00:20, John Snow wrote: On 11/13/2017 11:20 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all. There are three qmp commands, needed to implement external backup API. Using these three

Re: [Qemu-block] [Qemu-devel] [PATCH v8 02/14] block/dirty-bitmap: add locked version of bdrv_release_dirty_bitmap

2017-11-17 Thread Vladimir Sementsov-Ogievskiy
11.11.2017 01:52, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: It is needed to realize bdrv_dirty_bitmap_release_successor in the following patch. OK, but... Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c |