Re: [Qemu-block] Failed to get "consistent read" lock on a mirroring image

2017-11-20 Thread Kevin Wolf
[ Cc: qemu-block ] Am 20.11.2017 um 09:47 hat Fam Zheng geschrieben: > On Mon, 11/20 10:58, Han Han wrote: > > Hello, > > On qemu-2.10, I find 'qemu-img info' couldn't get the info of a mirroring > > image: > > # /usr/libexec/qemu-kvm -name A -machine pc,accel=kvm \ > > -vnc 0.0.0.0:1 \ > >

Re: [Qemu-block] [Qemu-devel] [PATCH v8 04/14] block/dirty-bitmap: add bdrv_dirty_bitmap_set_frozen

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
18.11.2017 02:46, John Snow wrote: On 11/17/2017 12:30 PM, Vladimir Sementsov-Ogievskiy wrote: 17.11.2017 20:20, John Snow wrote: On 11/17/2017 09:46 AM, Vladimir Sementsov-Ogievskiy wrote: 14.11.2017 02:32, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: Make

Re: [Qemu-block] [Qemu-devel] [PATCH v8 02/14] block/dirty-bitmap: add locked version of bdrv_release_dirty_bitmap

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 21:25, John Snow wrote: On 11/17/2017 03:07 AM, Vladimir Sementsov-Ogievskiy wrote: 11.11.2017 01:52, John Snow wrote: On 10/30/2017 12:32 PM, Vladimir Sementsov-Ogievskiy wrote: It is needed to realize bdrv_dirty_bitmap_release_successor in the following patch. OK, but...

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
17.11.2017 23:40, John Snow wrote: On 11/17/2017 01:25 PM, Kevin Wolf wrote: Am 17.11.2017 um 19:15 hat John Snow geschrieben: On 11/17/2017 10:01 AM, Max Reitz wrote: On 2017-11-17 13:30, Kevin Wolf wrote: Am 23.10.2017 um 11:29 hat Vladimir Sementsov-Ogievskiy geschrieben:

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Stefan Hajnoczi
On Sun, Nov 19, 2017 at 09:46:44PM -0500, Jeff Cody wrote: > diff --git a/include/qemu/coroutine_int.h b/include/qemu/coroutine_int.h > index 931cdc9..b071217 100644 > --- a/include/qemu/coroutine_int.h > +++ b/include/qemu/coroutine_int.h > @@ -56,6 +56,8 @@ struct Coroutine { > > int

Re: [Qemu-block] [PATCH] block/snapshot: dirty all dirty bitmaps on snapshot-switch

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
20.11.2017 12:51, Vladimir Sementsov-Ogievskiy wrote: 17.11.2017 23:40, John Snow wrote: On 11/17/2017 01:25 PM, Kevin Wolf wrote: Am 17.11.2017 um 19:15 hat John Snow geschrieben: On 11/17/2017 10:01 AM, Max Reitz wrote: On 2017-11-17 13:30, Kevin Wolf wrote: Am 23.10.2017 um 11:29 hat

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12 0/4] qmp dirty bitmap API

2017-11-20 Thread Denis V. Lunev
On 11/17/2017 06:10 AM, John Snow wrote: > > On 11/16/2017 03:17 AM, Vladimir Sementsov-Ogievskiy wrote: >> 16.11.2017 00:20, John Snow wrote: >>> On 11/13/2017 11:20 AM, Vladimir Sementsov-Ogievskiy wrote: Hi all. There are three qmp commands, needed to implement external backup

Re: [Qemu-block] [PATCH for-2.11 3/3] block: Error out on load_vm with active dirty bitmaps

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
20.11.2017 17:50, Kevin Wolf wrote: Loading a snapshot invalidates the bitmap. Just marking all blocks dirty is not a useful response in practice, instead the user needs to be aware that we switch to a completely different state. If they are okay with losing the dirty bitmap, they can just

[Qemu-block] [PATCH 1/7] qapi: add unmap to BlockDeviceStats

2017-11-20 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 20 include/block/accounting.h | 1 + block/qapi.c | 6 ++ 3 files changed, 27 insertions(+)

[Qemu-block] [PATCH 2/7] ide: account UNMAP (TRIM) operations

2017-11-20 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/ide/core.c | 12 1 file changed, 12 insertions(+) diff --git a/hw/ide/core.c b/hw/ide/core.c index 471d0c9..2e4dea7 100644 --- a/hw/ide/core.c +++

Re: [Qemu-block] [PATCH for-2.11 1/3] block: Add errp to bdrv_snapshot_goto()

2017-11-20 Thread Kevin Wolf
Am 20.11.2017 um 17:07 hat Vladimir Sementsov-Ogievskiy geschrieben: > 20.11.2017 17:50, Kevin Wolf wrote: > > Signed-off-by: Kevin Wolf > > --- > > include/block/snapshot.h | 3 ++- > > block/snapshot.c | 21 - > > qemu-img.c | 6

[Qemu-block] [PATCH 3/7] scsi: store unmap offset and nb_sectors in request struct

2017-11-20 Thread Anton Nefedov
it allows to report it in the error handler Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/scsi/scsi-disk.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/hw/scsi/scsi-disk.c

Re: [Qemu-block] [PATCH for-2.11 1/3] block: Add errp to bdrv_snapshot_goto()

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
20.11.2017 17:50, Kevin Wolf wrote: Signed-off-by: Kevin Wolf --- include/block/snapshot.h | 3 ++- block/snapshot.c | 21 - qemu-img.c | 6 +++--- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git

Re: [Qemu-block] [PATCH for-2.11 1/3] block: Add errp to bdrv_snapshot_goto()

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
20.11.2017 19:23, Kevin Wolf wrote: Am 20.11.2017 um 17:07 hat Vladimir Sementsov-Ogievskiy geschrieben: 20.11.2017 17:50, Kevin Wolf wrote: Signed-off-by: Kevin Wolf --- include/block/snapshot.h | 3 ++- block/snapshot.c | 21 -

[Qemu-block] [PATCH 7/7] qapi: query-blockstat: add driver specific file-posix stats

2017-11-20 Thread Anton Nefedov
A block driver can provide a callback to report driver-specific statistics. file-posix driver now reports discard statistics Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 37

[Qemu-block] [PATCH 6/7] file-posix: account discard operations

2017-11-20 Thread Anton Nefedov
This will help to identify how many of the user-issued discard operations (accounted on a device level) have actually suceeded down on the host file (even though the numbers will not be exactly the same if non-raw format driver is used (e.g. qcow2 sending metadata discards)). Signed-off-by: Anton

[Qemu-block] [PATCH 5/7] scsi: account unmap operations

2017-11-20 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy --- hw/scsi/scsi-disk.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index eca6a15..6c33418

[Qemu-block] [PATCH 4/7] scsi: move unmap error checking to the complete callback

2017-11-20 Thread Anton Nefedov
This will help to account the operation in the following commit. The difference is that we don't call scsi_disk_req_check_error() before the 1st discard iteration anymore. That function also checks if the request is cancelled, however it shouldn't get canceled until it yields in blk_aio()

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Paolo Bonzini
On 21/11/2017 00:08, Jeff Cody wrote: > @@ -34,6 +36,7 @@ void coroutine_fn co_aio_sleep_ns(AioContext *ctx, > QEMUClockType type, > CoSleepCB sleep_cb = { > .co = qemu_coroutine_self(), > }; > +if (sleep_cb.co->sleeping == 1 || sleep_cb.co->scheduled == 1) { > +

Re: [Qemu-block] [PATCH for-2.11 2/3] block: Add errp to bdrv_all_goto_snapshot()

2017-11-20 Thread Vladimir Sementsov-Ogievskiy
20.11.2017 17:50, Kevin Wolf wrote: Signed-off-by: Kevin Wolf --- include/block/snapshot.h | 3 ++- block/snapshot.c | 11 ++- migration/savevm.c | 6 +++--- 3 files changed, 11 insertions(+), 9 deletions(-) diff --git a/include/block/snapshot.h

[Qemu-block] [PATCH 0/7] discard blockstats

2017-11-20 Thread Anton Nefedov
qmp query-blockstats provides stats info for write/read/flush ops. Patches 1-5 implement the similar for discard (unmap) command for scsi and ide disks. Discard stat "unmap_ops / unmap_bytes" is supposed to account the ops that have completed without an error. However, discard operation is

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Paolo Bonzini
On 20/11/2017 03:46, Jeff Cody wrote: > Once a coroutine is "sleeping", the timer callback will either enter the > coroutine, or schedule it for the next AioContext if using iothreads. > > It is illegal to enter that coroutine while waiting for this timer > event and subsequent callback. This

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Paolo Bonzini
On 20/11/2017 23:35, Jeff Cody wrote: >> Is this a different "state" (in Stefan's parlance) than scheduled? In >> practice both means that someone may call qemu_(aio_)coroutine_enter >> concurrently, so you'd better not do it yourself. >> > It is slightly different; it is from sleeping with a

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Jeff Cody
On Mon, Nov 20, 2017 at 11:47:09PM +0100, Paolo Bonzini wrote: > On 20/11/2017 23:35, Jeff Cody wrote: > >> Is this a different "state" (in Stefan's parlance) than scheduled? In > >> practice both means that someone may call qemu_(aio_)coroutine_enter > >> concurrently, so you'd better not do it

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Jeff Cody
On Tue, Nov 21, 2017 at 12:13:46AM +0100, Paolo Bonzini wrote: > On 21/11/2017 00:08, Jeff Cody wrote: > > @@ -34,6 +36,7 @@ void coroutine_fn co_aio_sleep_ns(AioContext *ctx, > > QEMUClockType type, > > CoSleepCB sleep_cb = { > > .co = qemu_coroutine_self(), > > }; > > +if

Re: [Qemu-block] [Libguestfs] [Qemu-devel] [qemu-img] support for XVA

2017-11-20 Thread Gandalf Corvotempesta
I did something different, that will build a raw image directly from a xenserver export, on the fly. Compared the resulting file (via MD5) with xenmygrate.py and there is a match. Currently, this is the faster way to convert a XenServer image to a raw file. Don't need to wait for export, tar

Re: [Qemu-block] [PATCH 1/5] blockjob: do not allow coroutine double entry or entry-after-completion

2017-11-20 Thread Paolo Bonzini
On 20/11/2017 12:16, Stefan Hajnoczi wrote: > This raises questions about the ability to cancel sleep: > > 1. Does something depend on cancelling sleep? block_job_cancel does, but in practice the sleep time is so small (smaller than SLICE_TIME, which is 100 ms) that we probably don't care. I

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Jeff Cody
On Mon, Nov 20, 2017 at 11:30:39PM +0100, Paolo Bonzini wrote: > On 20/11/2017 03:46, Jeff Cody wrote: > > Once a coroutine is "sleeping", the timer callback will either enter the > > coroutine, or schedule it for the next AioContext if using iothreads. > > > > It is illegal to enter that

[Qemu-block] [PATCH v7 for-2.12 25/25] block/null: Generate filename even with latency-ns

2017-11-20 Thread Max Reitz
While we cannot represent the latency-ns option in a filename, it is not a significant option so not being able to should not stop us from generating a filename nonetheless. Signed-off-by: Max Reitz --- block/null.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)

[Qemu-block] [PATCH v7 for-2.12 06/25] block: Make path_combine() return the path

2017-11-20 Thread Max Reitz
Besides being safe for arbitrary path lengths, after some follow-up patches all callers will want a freshly allocated buffer anyway. In the meantime, path_combine_deprecated() is added which has the same interface as path_combine() had before this patch. All callers to that function will be

[Qemu-block] [PATCH v7 for-2.12 15/25] block/nfs: Implement bdrv_dirname()

2017-11-20 Thread Max Reitz
While the basic idea is obvious and could be handled by the default bdrv_dirname() implementation, we cannot generate a directory name if the gid or uid are set, so we have to explicitly return NULL in those cases. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia

[Qemu-block] [PATCH v7 for-2.12 13/25] quorum: Make bdrv_dirname() return NULL

2017-11-20 Thread Max Reitz
While the common implementation for bdrv_dirname() should return NULL for quorum BDSs already (because they do not have a file node and their exact_filename field should be empty), there is no reason not to make that explicit. Signed-off-by: Max Reitz Reviewed-by: Eric Blake

[Qemu-block] [PATCH v7 for-2.12 12/25] blkverify: Make bdrv_dirname() return NULL

2017-11-20 Thread Max Reitz
blkverify's BDSs have a file BDS, but we do not want this to be preferred over the raw node. There is no way to decide between the two (and not really a reason to, either), so just return NULL in blkverify's implementation of bdrv_dirname(). Signed-off-by: Max Reitz

[Qemu-block] [PATCH v7 for-2.12 07/25] block: bdrv_get_full_backing_filename_from_...'s ret. val.

2017-11-20 Thread Max Reitz
Make bdrv_get_full_backing_filename_from_filename() return an allocated string instead of placing the result in a caller-provided buffer. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- include/block/block.h | 7 +++ block.c | 51

[Qemu-block] [PATCH v7 for-2.12 05/25] block: Respect backing bs in bdrv_refresh_filename

2017-11-20 Thread Max Reitz
Basically, bdrv_refresh_filename() should respect all children of a BlockDriverState. However, generally those children are driver-specific, so this function cannot handle the general case. On the other hand, there are only few drivers which use other children than @file and @backing (that being

[Qemu-block] [PATCH v7 for-2.12 24/25] block/curl: Implement bdrv_refresh_filename()

2017-11-20 Thread Max Reitz
Signed-off-by: Max Reitz --- block/curl.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/block/curl.c b/block/curl.c index 11318a9a29..fe57223fda 100644 --- a/block/curl.c +++ b/block/curl.c @@ -957,6 +957,20 @@ static int64_t

[Qemu-block] [PATCH v7 for-2.12 20/25] block: Generically refresh runtime options

2017-11-20 Thread Max Reitz
Instead of having every block driver which implements bdrv_refresh_filename() copy all of the significant runtime options over to bs->full_open_options, implement this process generically in bdrv_refresh_filename(). This patch only adds this new generic implementation, it does not remove the old

[Qemu-block] [PATCH v7 for-2.12 23/25] block: Fix FIXME from "Add BDS.backing_overridden"

2017-11-20 Thread Max Reitz
Said commit introduced a FIXME stating that bdrv_open_backing_file() should set bs->backing_overridden to true not only if the file.filename option was set, but if the "options" QDict contained any option that is significant for any node in the BDS tree emerging from the backing BDS. This behavior

[Qemu-block] [PATCH v7 for-2.12 22/25] block: Do not copy exact_filename from format file

2017-11-20 Thread Max Reitz
If the a format BDS's file BDS is in turn a format BDS, we cannot simply use the same filename, because when opening a BDS tree based on a filename alone, qemu will create only one format node on top of one protocol node (disregarding a potential backing file). Signed-off-by: Max Reitz

Re: [Qemu-block] [PATCH] block: Close a BlockDriverState completely even when bs->drv is NULL

2017-11-20 Thread Max Reitz
On 2017-11-06 15:53, Alberto Garcia wrote: > bdrv_close() skips much of its logic when bs->drv is NULL. This is > fine when we're closing a BlockDriverState that has just been created > (because e.g the initialization process failed), but it's not enough > in other cases. > > For example, when a

[Qemu-block] [PATCH v7 for-2.12 02/25] block: Use children list in bdrv_refresh_filename

2017-11-20 Thread Max Reitz
bdrv_refresh_filename() should invoke itself recursively on all children, not just on file. With that change, we can remove the manual invocations in blkverify, quorum, commit, and mirror. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- block.c

[Qemu-block] [PATCH v7 for-2.12 17/25] iotests: Add quorum case to test 110

2017-11-20 Thread Max Reitz
Test 110 tests relative backing filenames for complex BDS trees. Now that the originally supposedly failing test passes, let us add a new failing test: Quorum can never work automatically (without detecting whether all child nodes have the same base directory, but that would be rather

[Qemu-block] [PATCH v7 for-2.12 14/25] block/nbd: Make bdrv_dirname() return NULL

2017-11-20 Thread Max Reitz
The generic bdrv_dirname() implementation would be able to generate some form of directory name for many NBD nodes, but it would be always wrong. Therefore, we have to explicitly make it an error (until NBD has some form of specification for export paths, if it ever will). Signed-off-by: Max

[Qemu-block] [PATCH v7 for-2.12 11/25] block: Add bdrv_dirname()

2017-11-20 Thread Max Reitz
This function may be implemented by block drivers to derive a directory name from a BDS. Concatenating this g_free()-able string with a relative filename must result in a valid (not necessarily existing) filename, so this is a function that should generally be not implemented by format drivers,

[Qemu-block] [PATCH v7 for-2.12 16/25] block: Use bdrv_dirname() for relative filenames

2017-11-20 Thread Max Reitz
bdrv_get_full_backing_filename_from_filename() breaks down when it comes to JSON filenames. Using bdrv_dirname() as the basis is better because since we have BDS, we can descend through the BDS tree to the protocol layer, which gives us a greater probability of finding a non-JSON name; also,

Re: [Qemu-block] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-20 Thread Max Reitz
On 2017-11-17 20:04, Eric Blake wrote: > The contents of a qcow2 bitmap are rounded up to a size that > matches the number of bits available for the granularity, but > that granularity differs for 32-bit hosts (our default 64k > cluster allows for 2M bitmap coverage per 'long') and 64-bit > hosts

[Qemu-block] [PATCH v7 for-2.12 21/25] block: Purify .bdrv_refresh_filename()

2017-11-20 Thread Max Reitz
Currently, BlockDriver.bdrv_refresh_filename() is supposed to both refresh the filename (BDS.exact_filename) and set BDS.full_open_options. Now that we have generic code in the central bdrv_refresh_filename() for creating BDS.full_open_options, we can drop the latter part from all

[Qemu-block] [PATCH v7 for-2.12 00/25] block: Fix some filename generation issues

2017-11-20 Thread Max Reitz
I'm sparing myself writing this cover letter again, and I'll just give you a link to the previous version: http://lists.nongnu.org/archive/html/qemu-block/2017-09/msg01030.html The only difference is that I dropped patch 16 which added a QAPI @base-directory option for any node that could be

[Qemu-block] [PATCH v7 for-2.12 01/25] block/mirror: Small absolute-paths simplification

2017-11-20 Thread Max Reitz
When invoking drive-mirror in absolute-paths mode, the target's backing BDS is assigned to it in mirror_exit(). The current logic only does so if the target does not have that backing BDS already; but it actually cannot have a backing BDS at all (the BDS is opened with O_NO_BACKING in

[Qemu-block] [PATCH v7 for-2.12 03/25] block: Add BDS.backing_overridden

2017-11-20 Thread Max Reitz
If the backing file is overridden, this most probably does change the guest-visible data of a BDS. Therefore, we will need to consider this in bdrv_refresh_filename(). Adding a new field to the BDS is not nice, but it is very simple and exactly keeps track of whether the backing file has been

[Qemu-block] [PATCH v7 for-2.12 10/25] block: Fix bdrv_find_backing_image()

2017-11-20 Thread Max Reitz
bdrv_find_backing_image() should use bdrv_get_full_backing_filename() or bdrv_make_absolute_filename() instead of trying to do what those functions do by itself. path_combine_deprecated() can now be dropped, so let's do that. Signed-off-by: Max Reitz --- block.c | 30

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-20 Thread Eric Blake
On 11/17/2017 04:59 PM, John Snow wrote: >>> So for this test, the easiest solution is to filter out the >>> actual hash value. Broken in commit 4096974e. >> >> Of course, if Kevin sends a v2 pull, it's probably better to just squash >> this in to my original testsuite change (since a v2 would

[Qemu-block] [PATCH v2 for-2.11 0/4] Fix segfault in blockjob race condition

2017-11-20 Thread Jeff Cody
Changes from v1 -> v2: Patch 1: Updated docs in blockjob_int.h (Thanks Stefan) Patch 2/3: Squashed, and used const char * to hold the __func__ name of the original scheduler (Thanks Paolo) Patch 4: Unchanged. Patch 5: Dropped qcow format for the test, it was so slow the test times

[Qemu-block] [PATCH v2 for-2.11 2/4] coroutine: abort if we try to schedule or enter a pending coroutine

2017-11-20 Thread Jeff Cody
The previous patch fixed a race condition, in which there were coroutines being executing doubly, or after coroutine deletion. We can detect common scenarios when this happens, and print an error message and abort before we corrupt memory / data, or segfault. This patch will abort if an attempt

[Qemu-block] [PATCH v2 for-2.11 1/4] blockjob: do not allow coroutine double entry or entry-after-completion

2017-11-20 Thread Jeff Cody
When block_job_sleep_ns() is called, the co-routine is scheduled for future execution. If we allow the job to be re-entered prior to the scheduled time, we present a race condition in which a coroutine can be entered recursively, or even entered after the coroutine is deleted. The job->busy flag

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.11 0/3] block: Error out on load_vm with active dirty bitmaps

2017-11-20 Thread John Snow
On 11/20/2017 09:50 AM, Kevin Wolf wrote: > Following the discussing at > https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg03572.html > this implements the error return for loading a snapshot while dirty > bitmaps are active. > > Kevin Wolf (3): > block: Add errp to

[Qemu-block] [PATCH v2 for-2.11 4/4] qemu-iotest: add test for blockjob coroutine race condition

2017-11-20 Thread Jeff Cody
Signed-off-by: Jeff Cody --- tests/qemu-iotests/200 | 99 ++ tests/qemu-iotests/200.out | 14 +++ tests/qemu-iotests/group | 1 + 3 files changed, 114 insertions(+) create mode 100755 tests/qemu-iotests/200 create mode

[Qemu-block] [PATCH v2 for-2.11 3/4] qemu-iotests: add option in common.qemu for mismatch only

2017-11-20 Thread Jeff Cody
Add option to echo response to QMP / HMP command only on mismatch. Useful for ignore all normal responses, but catching things like segfaults. Signed-off-by: Jeff Cody --- tests/qemu-iotests/common.qemu | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git

[Qemu-block] [PATCH v7 for-2.12 09/25] block: Add bdrv_make_absolute_filename()

2017-11-20 Thread Max Reitz
This is a general function for making a filename that is relative to a certain BDS absolute. It calls bdrv_get_full_backing_filename_from_filename() for now, but that will be changed in a follow-up patch. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia

[Qemu-block] [PATCH v7 for-2.12 19/25] block: Add BlockDriver.bdrv_gather_child_options

2017-11-20 Thread Max Reitz
Some follow-up patches will rework the way bs->full_open_options is refreshed in bdrv_refresh_filename(). The new implementation will remove the need for the block drivers' bdrv_refresh_filename() implementations to set bs->full_open_options; instead, it will be generic and use static information

[Qemu-block] [PATCH v7 for-2.12 04/25] iotests: Drop explicit base blockdev in 191

2017-11-20 Thread Max Reitz
Overriding the backing image should result in a json:{} pseudo-filename. Then, you can no longer use the commit block job with filename parameters. Therefore, do not explicitly add the base and override the middle image in iotest 191, since we do not need to anyway. This will allow us to

[Qemu-block] [PATCH v7 for-2.12 08/25] block: bdrv_get_full_backing_filename's ret. val.

2017-11-20 Thread Max Reitz
Make bdrv_get_full_backing_filename() return an allocated string instead of placing the result in a caller-provided buffer. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- include/block/block.h | 3 +-- block.c | 47

Re: [Qemu-block] [PATCH for-2.11] iotests: Fix 176 on 32-bit host

2017-11-20 Thread Max Reitz
On 2017-11-17 20:04, Eric Blake wrote: > The contents of a qcow2 bitmap are rounded up to a size that > matches the number of bits available for the granularity, but > that granularity differs for 32-bit hosts (our default 64k > cluster allows for 2M bitmap coverage per 'long') and 64-bit > hosts

Re: [Qemu-block] Failed to get "consistent read" lock on a mirroring image

2017-11-20 Thread Kevin Wolf
Am 20.11.2017 um 13:39 hat Eric Blake geschrieben: > On 11/20/2017 04:37 AM, Kevin Wolf wrote: > > [ Cc: qemu-block ] > > > > Am 20.11.2017 um 09:47 hat Fam Zheng geschrieben: > >> On Mon, 11/20 10:58, Han Han wrote: > >>> Hello, > >>> On qemu-2.10, I find 'qemu-img info' couldn't get the info of

Re: [Qemu-block] [PATCH 2/5] coroutine: abort if we try to enter coroutine scheduled for another ctx

2017-11-20 Thread Jeff Cody
On Mon, Nov 20, 2017 at 11:28:26AM +, Stefan Hajnoczi wrote: > On Sun, Nov 19, 2017 at 09:46:43PM -0500, Jeff Cody wrote: > > diff --git a/include/qemu/coroutine_int.h b/include/qemu/coroutine_int.h > > index cb98892..931cdc9 100644 > > --- a/include/qemu/coroutine_int.h > > +++

Re: [Qemu-block] [PATCH 3/5] coroutines: abort if we try to enter a still-sleeping coroutine

2017-11-20 Thread Jeff Cody
On Mon, Nov 20, 2017 at 11:43:34AM +, Stefan Hajnoczi wrote: > On Sun, Nov 19, 2017 at 09:46:44PM -0500, Jeff Cody wrote: > > diff --git a/include/qemu/coroutine_int.h b/include/qemu/coroutine_int.h > > index 931cdc9..b071217 100644 > > --- a/include/qemu/coroutine_int.h > > +++

Re: [Qemu-block] Failed to get "consistent read" lock on a mirroring image

2017-11-20 Thread Eric Blake
On 11/20/2017 04:37 AM, Kevin Wolf wrote: > [ Cc: qemu-block ] > > Am 20.11.2017 um 09:47 hat Fam Zheng geschrieben: >> On Mon, 11/20 10:58, Han Han wrote: >>> Hello, >>> On qemu-2.10, I find 'qemu-img info' couldn't get the info of a mirroring >>> image: >> >> Cc Kevin. Looks like -U here is

Re: [Qemu-block] [PATCH 1/5] blockjob: do not allow coroutine double entry or entry-after-completion

2017-11-20 Thread Stefan Hajnoczi
On Sun, Nov 19, 2017 at 09:46:42PM -0500, Jeff Cody wrote: > --- a/blockjob.c > +++ b/blockjob.c > @@ -291,10 +291,10 @@ void block_job_start(BlockJob *job) > { > assert(job && !block_job_started(job) && job->paused && > job->driver && job->driver->start); > -job->co =

Re: [Qemu-block] [PATCH 2/5] coroutine: abort if we try to enter coroutine scheduled for another ctx

2017-11-20 Thread Stefan Hajnoczi
On Sun, Nov 19, 2017 at 09:46:43PM -0500, Jeff Cody wrote: > diff --git a/include/qemu/coroutine_int.h b/include/qemu/coroutine_int.h > index cb98892..931cdc9 100644 > --- a/include/qemu/coroutine_int.h > +++ b/include/qemu/coroutine_int.h > @@ -53,6 +53,9 @@ struct Coroutine { > > /* Only

Re: [Qemu-block] [PATCH for-2.11 0/2] Fix 'qemu-img info' on mirror target

2017-11-20 Thread Fam Zheng
On Mon, 11/20 15:11, Kevin Wolf wrote: > There is no reason to forbid 'qemu-img info' for an image that is used > as a mirror target (or intermediate image of a commit job) at the same > time. > > This fixes the problem reported on qemu-discuss: >

Re: [Qemu-block] [PATCH for-2.11] qcow2: fix image corruption on commit with persistent snapshot

2017-11-20 Thread Kevin Wolf
Am 17.11.2017 um 19:46 hat Eric Blake geschrieben: > > -- and I'm not really a fan of testing this on every possible > > architecture and then adding different reference outputs. > > > > Therefore, the best fix is probably to just filter the hashes out (you > > don't need the exact value anyway,

Re: [Qemu-block] [PATCH for-2.11 2/2] block: Don't request I/O permission with BDRV_O_NO_IO

2017-11-20 Thread Alberto Garcia
On Mon 20 Nov 2017 03:11:41 PM CET, Kevin Wolf wrote: > 'qemu-img info' makes sense even when BLK_PERM_CONSISTENT_READ cannot be > granted because of a block job in a running qemu process. It already > sets BDRV_O_NO_IO to indicate that it doesn't access the guest visible > data at all. > > Check

[Qemu-block] [PATCH for-2.11 2/2] block: Don't request I/O permission with BDRV_O_NO_IO

2017-11-20 Thread Kevin Wolf
'qemu-img info' makes sense even when BLK_PERM_CONSISTENT_READ cannot be granted because of a block job in a running qemu process. It already sets BDRV_O_NO_IO to indicate that it doesn't access the guest visible data at all. Check the BDRV_O_NO_IO flags in blk_new_open(), so that I/O related

[Qemu-block] [PATCH for-2.11 0/2] Fix 'qemu-img info' on mirror target

2017-11-20 Thread Kevin Wolf
There is no reason to forbid 'qemu-img info' for an image that is used as a mirror target (or intermediate image of a commit job) at the same time. This fixes the problem reported on qemu-discuss: https://lists.gnu.org/archive/html/qemu-discuss/2017-11/msg00039.html Kevin Wolf (2): block:

[Qemu-block] [PATCH for-2.11 1/2] block: Don't use BLK_PERM_CONSISTENT_READ for format probing

2017-11-20 Thread Kevin Wolf
For format probing, we don't really care whether all of the image content is consistent. The only thing we're looking at is the image header, and specifically the magic numbers that are expected to never change, no matter how inconsistent the guest visible disk content is. Therefore, don't

Re: [Qemu-block] [Qemu-devel] [PATCH v2 4/4] iotests: 030: add compressed block-stream test

2017-11-20 Thread Stefan Hajnoczi
On Thu, Nov 16, 2017 at 07:54:58PM +0300, Anton Nefedov wrote: > Signed-off-by: Anton Nefedov > --- > tests/qemu-iotests/030 | 66 > +- > tests/qemu-iotests/030.out | 4 +-- > 2 files changed, 67 insertions(+), 3

[Qemu-block] [PATCH for-2.11 1/3] block: Add errp to bdrv_snapshot_goto()

2017-11-20 Thread Kevin Wolf
Signed-off-by: Kevin Wolf --- include/block/snapshot.h | 3 ++- block/snapshot.c | 21 - qemu-img.c | 6 +++--- 3 files changed, 21 insertions(+), 9 deletions(-) diff --git a/include/block/snapshot.h b/include/block/snapshot.h index

[Qemu-block] [PATCH for-2.11 0/3] block: Error out on load_vm with active dirty bitmaps

2017-11-20 Thread Kevin Wolf
Following the discussing at https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg03572.html this implements the error return for loading a snapshot while dirty bitmaps are active. Kevin Wolf (3): block: Add errp to bdrv_snapshot_goto() block: Add errp to bdrv_all_goto_snapshot() block:

[Qemu-block] [PATCH for-2.11 3/3] block: Error out on load_vm with active dirty bitmaps

2017-11-20 Thread Kevin Wolf
Loading a snapshot invalidates the bitmap. Just marking all blocks dirty is not a useful response in practice, instead the user needs to be aware that we switch to a completely different state. If they are okay with losing the dirty bitmap, they can just explicitly delete it. This effectively

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/4] qcow2: multiple clusters write compressed

2017-11-20 Thread Stefan Hajnoczi
On Thu, Nov 16, 2017 at 07:54:55PM +0300, Anton Nefedov wrote: > From: Pavel Butsykin > > At the moment, qcow2_co_pwritev_compressed can process the requests size > less than or equal to one cluster. This patch added possibility to write > compressed data in the QCOW2

Re: [Qemu-block] [Qemu-devel] [PULL 00/25] Block layer patches for 2.11.0-rc2

2017-11-20 Thread Peter Maydell
On 17 November 2017 at 18:16, Kevin Wolf wrote: > The following changes since commit fec035a53fa15c4c8c4e62bfef56a35df4161e38: > > Merge remote-tracking branch 'remotes/kraxel/tags/ui-20171117-pull-request' > into staging (2017-11-17 10:18:41 +) > > are available in the

Re: [Qemu-block] [PATCH v2 3/4] block-stream: add compress option

2017-11-20 Thread Stefan Hajnoczi
On Thu, Nov 16, 2017 at 07:54:57PM +0300, Anton Nefedov wrote: > It might be useful to compress images during block-stream; > this way the user can merge compressed images of a backing chain and > the result will remain compressed. > > Signed-off-by: Anton Nefedov >

Re: [Qemu-block] [PATCH v2 2/4] block: support compressed write for copy-on-read

2017-11-20 Thread Stefan Hajnoczi
On Thu, Nov 16, 2017 at 07:54:56PM +0300, Anton Nefedov wrote: > Signed-off-by: Anton Nefedov > Reviewed-by: Max Reitz > --- > block/io.c | 23 +-- > block/trace-events | 2 +- > 2 files changed, 18 insertions(+), 7

Re: [Qemu-block] [Qemu-devel] [PATCH v2 1/4] qcow2: multiple clusters write compressed

2017-11-20 Thread Denis V. Lunev
On 11/20/2017 05:53 PM, Stefan Hajnoczi wrote: > On Thu, Nov 16, 2017 at 07:54:55PM +0300, Anton Nefedov wrote: >> From: Pavel Butsykin >> >> At the moment, qcow2_co_pwritev_compressed can process the requests size >> less than or equal to one cluster. This patch added

Re: [Qemu-block] [PATCH for-2.11 0/3] block: Error out on load_vm with active dirty bitmaps

2017-11-20 Thread Denis V. Lunev
On 11/20/2017 05:50 PM, Kevin Wolf wrote: > Following the discussing at > https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg03572.html > this implements the error return for loading a snapshot while dirty > bitmaps are active. > > Kevin Wolf (3): > block: Add errp to bdrv_snapshot_goto()

Re: [Qemu-block] [Qemu-devel] [PULL 00/25] Block layer patches for 2.11.0-rc2

2017-11-20 Thread Kevin Wolf
Am 20.11.2017 um 15:53 hat Peter Maydell geschrieben: > On 17 November 2017 at 18:16, Kevin Wolf wrote: > > The following changes since commit fec035a53fa15c4c8c4e62bfef56a35df4161e38: > > > > Merge remote-tracking branch > > 'remotes/kraxel/tags/ui-20171117-pull-request'