Re: [Qemu-block] [Qemu-devel] [PATCH v4 11/40] hw/xen: Use the IEC binary prefix definitions

2018-06-13 Thread Philippe Mathieu-Daudé
On 06/13/2018 04:31 PM, Richard Henderson wrote: > On 06/13/2018 02:13 AM, Eric Blake wrote: >> Or spell it UINT64_C(1) if you don't want a cast. > > Not unsigned is what I want most. I used both of your suggestions, but now new format string errors appeared due to ram_addr_t being unsigned, so

Re: [Qemu-block] [Qemu-devel] [PATCH v4 11/40] hw/xen: Use the IEC binary prefix definitions

2018-06-13 Thread Richard Henderson
On 06/13/2018 02:13 AM, Eric Blake wrote: > Or spell it UINT64_C(1) if you don't want a cast. Not unsigned is what I want most. r~

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/8] discard blockstats

2018-06-13 Thread no-reply
Hi, This series failed docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. Type: series Message-id: 1528911866-37489-1-git-send-email-anton.nefe...@virtuozzo.com Subject: [Qemu-devel]

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/8] discard blockstats

2018-06-13 Thread no-reply
Hi, This series failed build test on s390x host. Please find the details below. Type: series Message-id: 1528911866-37489-1-git-send-email-anton.nefe...@virtuozzo.com Subject: [Qemu-devel] [PATCH v3 0/8] discard blockstats === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked

Re: [Qemu-block] [Qemu-devel] [PATCH 0/3] block: Remove deprecated -drive options

2018-06-13 Thread Cole Robinson
On 06/13/2018 08:34 AM, Kevin Wolf wrote: > We deprecated a bunch of -drive options in 2.10, so let's remove them > for 3.0. > > Kevin Wolf (3): > block: Remove deprecated -drive geometry options > block: Remove deprecated -drive option addr > block: Remove deprecated -drive option serial >

[Qemu-block] [PATCH v5 12/14] block/mirror: Add active mirroring

2018-06-13 Thread Max Reitz
This patch implements active synchronous mirroring. In active mode, the passive mechanism will still be in place and is used to copy all initially dirty clusters off the source disk; but every write request will write data both to the source and the target disk, so the source cannot be dirtied

[Qemu-block] [PATCH v5 10/14] block/mirror: Add MirrorBDSOpaque

2018-06-13 Thread Max Reitz
This will allow us to access the block job data when the mirror block driver becomes more complex. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng --- block/mirror.c | 12 1 file changed, 12 insertions(+) diff --git a/block/mirror.c b/block/mirror.c index fa94fd9d17..f060bd8bb4

[Qemu-block] [PATCH v5 14/14] iotests: Add test for active mirroring

2018-06-13 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: Alberto Garcia --- tests/qemu-iotests/151 | 120 + tests/qemu-iotests/151.out | 5 ++ tests/qemu-iotests/group | 1 + 3 files changed, 126 insertions(+) create mode 100755

[Qemu-block] [PATCH v5 08/14] test-hbitmap: Add non-advancing iter_next tests

2018-06-13 Thread Max Reitz
Add a function that wraps hbitmap_iter_next() and always calls it in non-advancing mode first, and in advancing mode next. The result should always be the same. By using this function everywhere we called hbitmap_iter_next() before, we should get good test coverage for non-advancing

[Qemu-block] [PATCH v5 07/14] hbitmap: Add @advance param to hbitmap_iter_next()

2018-06-13 Thread Max Reitz
This new parameter allows the caller to just query the next dirty position without moving the iterator. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: John Snow --- include/qemu/hbitmap.h | 5 - block/backup.c | 2 +- block/dirty-bitmap.c | 2 +-

[Qemu-block] [PATCH v5 13/14] block/mirror: Add copy mode QAPI interface

2018-06-13 Thread Max Reitz
This patch allows the user to specify whether to use active or only background mode for mirror block jobs. Currently, this setting will remain constant for the duration of the entire block job. Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia --- qapi/block-core.json | 11 +--

[Qemu-block] [PATCH v5 11/14] job: Add job_progress_increase_remaining()

2018-06-13 Thread Max Reitz
Signed-off-by: Max Reitz --- include/qemu/job.h | 15 +++ job.c | 5 + 2 files changed, 20 insertions(+) diff --git a/include/qemu/job.h b/include/qemu/job.h index 1d820530fa..18c9223e31 100644 --- a/include/qemu/job.h +++ b/include/qemu/job.h @@ -335,6 +335,21 @@

[Qemu-block] [PATCH v5 02/14] block/mirror: Convert to coroutines

2018-06-13 Thread Max Reitz
In order to talk to the source BDS (and maybe in the future to the target BDS as well) directly, we need to convert our existing AIO requests into coroutine I/O requests. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng --- block/mirror.c | 152 +

[Qemu-block] [PATCH v5 09/14] block/dirty-bitmap: Add bdrv_dirty_iter_next_area

2018-06-13 Thread Max Reitz
This new function allows to look for a consecutively dirty area in a dirty bitmap. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: John Snow --- include/block/dirty-bitmap.h | 2 ++ block/dirty-bitmap.c | 55 2 files changed, 57

[Qemu-block] [PATCH v5 01/14] block/mirror: Pull out mirror_perform()

2018-06-13 Thread Max Reitz
When converting mirror's I/O to coroutines, we are going to need a point where these coroutines are created. mirror_perform() is going to be that point. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Jeff Cody Reviewed-by: Alberto

[Qemu-block] [PATCH v5 03/14] block/mirror: Use CoQueue to wait on in-flight ops

2018-06-13 Thread Max Reitz
Attach a CoQueue to each in-flight operation so if we need to wait for any we can use it to wait instead of just blindly yielding and hoping for some operation to wake us. A later patch will use this infrastructure to allow requests accessing the same area of the virtual disk to specifically wait

[Qemu-block] [PATCH v5 06/14] block: Generalize should_update_child() rule

2018-06-13 Thread Max Reitz
Currently, bdrv_replace_node() refuses to create loops from one BDS to itself if the BDS to be replaced is the backing node of the BDS to replace it: Say there is a node A and a node B. Replacing B by A means making all references to B point to A. If B is a child of A (i.e. A has a reference to

[Qemu-block] [PATCH v5 05/14] block/mirror: Use source as a BdrvChild

2018-06-13 Thread Max Reitz
With this, the mirror_top_bs is no longer just a technically required node in the BDS graph but actually represents the block job operation. Also, drop MirrorBlockJob.source, as we can reach it through mirror_top_bs->backing. Signed-off-by: Max Reitz Reviewed-by: Fam Zheng Reviewed-by: Alberto

[Qemu-block] [PATCH v5 04/14] block/mirror: Wait for in-flight op conflicts

2018-06-13 Thread Max Reitz
This patch makes the mirror code differentiate between simply waiting for any operation to complete (mirror_wait_for_free_in_flight_slot()) and specifically waiting for all operations touching a certain range of the virtual disk to complete (mirror_wait_on_conflicts()). Signed-off-by: Max Reitz

[Qemu-block] [PATCH v5 00/14] block/mirror: Add active-sync mirroring

2018-06-13 Thread Max Reitz
This series implements an active and synchronous mirroring mode. You can read the cover letter of v4 here (I don’t like to copy-paste that because people who reviewed previous versions know it already and this saves them from having to look out for potential changes):

[Qemu-block] [PATCH v3 8/8] qapi: query-blockstat: add driver specific file-posix stats

2018-06-13 Thread Anton Nefedov
A block driver can provide a callback to report driver-specific statistics. file-posix driver now reports discard statistics Signed-off-by: Anton Nefedov --- qapi/block-core.json | 39 +++ include/block/block.h | 1 + include/block/block_int.h | 1

[Qemu-block] [PATCH v3 6/8] scsi: account unmap operations

2018-06-13 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- hw/scsi/scsi-disk.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c index 0b68a8d..ed72662 100644 ---

[Qemu-block] [PATCH v3 7/8] file-posix: account discard operations

2018-06-13 Thread Anton Nefedov
This will help to identify how many of the user-issued discard operations (accounted on a device level) have actually suceeded down on the host file (even though the numbers will not be exactly the same if non-raw format driver is used (e.g. qcow2 sending metadata discards)). Signed-off-by: Anton

[Qemu-block] [PATCH v3 0/8] discard blockstats

2018-06-13 Thread Anton Nefedov
new in v3: - based on qapi series: (http://lists.nongnu.org/archive/html/qemu-devel/2018-06/msg03638.html) - qapi 'Since' clauses changed 2.12 -> 3.0 (RB tags not stripped) - patch 3 rebase: ide_issue_trim_cb invalid path has changed, so block_acct_invalid() is moved to

[Qemu-block] [PATCH v3 5/8] scsi: move unmap error checking to the complete callback

2018-06-13 Thread Anton Nefedov
This will help to account the operation in the following commit. The difference is that we don't call scsi_disk_req_check_error() before the 1st discard iteration anymore. That function also checks if the request is cancelled, however it shouldn't get canceled until it yields in blk_aio()

[Qemu-block] [PATCH v3 1/8] qapi: group BlockDeviceStats fields

2018-06-13 Thread Anton Nefedov
Make the stat fields definition slightly more readable. Also reorder total_time_ns stats read-write-flush as done elsewhere. Cosmetic change only. Signed-off-by: Anton Nefedov --- qapi/block-core.json | 34 +++--- 1 file changed, 23 insertions(+), 11 deletions(-)

[Qemu-block] [PATCH v3 4/8] scsi: store unmap offset and nb_sectors in request struct

2018-06-13 Thread Anton Nefedov
it allows to report it in the error handler Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia --- hw/scsi/scsi-disk.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/hw/scsi/scsi-disk.c b/hw/scsi/scsi-disk.c

[Qemu-block] [PATCH v3 3/8] ide: account UNMAP (TRIM) operations

2018-06-13 Thread Anton Nefedov
Signed-off-by: Anton Nefedov --- hw/ide/core.c | 12 1 file changed, 12 insertions(+) diff --git a/hw/ide/core.c b/hw/ide/core.c index 2c62efc..352429b 100644 --- a/hw/ide/core.c +++ b/hw/ide/core.c @@ -440,6 +440,14 @@ static void ide_issue_trim_cb(void *opaque, int ret)

[Qemu-block] [PATCH v3 2/8] qapi: add unmap to BlockDeviceStats

2018-06-13 Thread Anton Nefedov
Signed-off-by: Anton Nefedov Reviewed-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Alberto Garcia Reviewed-by: Eric Blake --- qapi/block-core.json | 29 +++-- include/block/accounting.h | 1 + block/qapi.c | 6 ++ 3 files changed, 30

Re: [Qemu-block] [PATCH v0 2/2] qmp: add block-set-copy-on-read command

2018-06-13 Thread Max Reitz
On 2018-06-13 18:02, Eric Blake wrote: > On 06/13/2018 10:47 AM, Denis Plotnikov wrote: >> The command enables/disables copy-on-read mode for VM's disk while >> VM is running. >> >> This is needed when using external disk readers to shape access pattern >> to the disk backend. >> >> Signed-off-by:

Re: [Qemu-block] [PATCH v0 2/2] qmp: add block-set-copy-on-read command

2018-06-13 Thread Eric Blake
On 06/13/2018 10:47 AM, Denis Plotnikov wrote: The command enables/disables copy-on-read mode for VM's disk while VM is running. This is needed when using external disk readers to shape access pattern to the disk backend. Signed-off-by: Denis Plotnikov --- Deferring thoughts on the actual

[Qemu-block] [PATCH v0 1/2] block: check for read-only on copy-on-read setting

2018-06-13 Thread Denis Plotnikov
We should always check whether block device is in the read-only state before enabling copy-on-read. The patch embeds the check in the copy-on-read setter. Signed-off-by: Denis Plotnikov --- block.c | 4 +--- block/io.c| 10 -- block/stream.c| 5 +++--

[Qemu-block] [PATCH v0 2/2] qmp: add block-set-copy-on-read command

2018-06-13 Thread Denis Plotnikov
The command enables/disables copy-on-read mode for VM's disk while VM is running. This is needed when using external disk readers to shape access pattern to the disk backend. Signed-off-by: Denis Plotnikov --- blockdev.c | 38 ++

[Qemu-block] [PATCH v0 0/2] enable/disable copy-on-read via qmp

2018-06-13 Thread Denis Plotnikov
In some cases there is a need to perform a special access pattern to a disk backend. This may be done by providing disk reading access to external readers. In this case there should be the ability to enable/disable the copy-on-read mode for the disk during VM runtime. By the moment, there is no

Re: [Qemu-block] [Qemu-devel] [PATCH 3/3] block: Remove deprecated -drive option serial

2018-06-13 Thread Jeff Cody
On Wed, Jun 13, 2018 at 04:18:17PM +0200, Markus Armbruster wrote: > Kevin Wolf writes: > > > The -drive option serial was deprecated in QEMU 2.10. It's time to > > remove it. > > > > Tests need to be updated to set the serial number with -global instead > > of using the -drive option. > > > >

Re: [Qemu-block] [PATCH 2/3] block: Remove deprecated -drive option addr

2018-06-13 Thread Jeff Cody
On Wed, Jun 13, 2018 at 02:34:57PM +0200, Kevin Wolf wrote: > The -drive option addr was deprecated in QEMU 2.10. It's time to remove > it. > > Signed-off-by: Kevin Wolf Reviewed-by: Jeff Cody > --- > include/sysemu/blockdev.h | 1 - > blockdev.c| 17 + >

Re: [Qemu-block] [PATCH 1/3] block: Remove deprecated -drive geometry options

2018-06-13 Thread Jeff Cody
On Wed, Jun 13, 2018 at 02:34:56PM +0200, Kevin Wolf wrote: > The -drive options cyls, heads, secs and trans were deprecated in > QEMU 2.10. It's time to remove them. > > hd-geo-test tested both the old version with geometry options in -drive > and the new one with -device. Therefore the code

Re: [Qemu-block] [Qemu-devel] [PATCH 12/18] block-qdict: Clean up qdict_crumple() a bit

2018-06-13 Thread Markus Armbruster
Kevin Wolf writes: > Am 12.06.2018 um 14:58 hat Markus Armbruster geschrieben: >> When you mix scalar and non-scalar keys, whether you get an "already >> set as scalar" or an "already set as dict" error depends on qdict >> iteration order. Neither message makes much sense. Replace by >>

[Qemu-block] [PULL 1/1] block: Ignore generated job QAPI files

2018-06-13 Thread Jeff Cody
From: Eric Blake Commit bf42508f introduced new generated files; make sure they don't get accidentally committed from an in-tree build. Signed-off-by: Eric Blake Reviewed-by: Jeff Cody Reviewed-by: Max Reitz Message-id: 20180531212435.165261-1-ebl...@redhat.com Signed-off-by: Jeff Cody ---

[Qemu-block] [PULL 0/1] Block patches

2018-06-13 Thread Jeff Cody
The following changes since commit 2ab09bf2f9f55b9fb8d2de6eb2ba2a8570e268e2: Merge remote-tracking branch 'remotes/kraxel/tags/usb-20180612-pull-request' into staging (2018-06-12 15:34:34 +0100) are available in the git repository at: git://github.com/codyprime/qemu-kvm-jtc.git

Re: [Qemu-block] [PATCH 0/2] jobs: minor doc fixups

2018-06-13 Thread Kevin Wolf
Am 07.06.2018 um 01:02 hat John Snow geschrieben: > One is from my manual completion series, > the other is from the recent refactor. Thanks, applied to the block branch. Kevin

Re: [Qemu-block] [Qemu-devel] [PATCH 3/3] block: Remove deprecated -drive option serial

2018-06-13 Thread Markus Armbruster
Kevin Wolf writes: > The -drive option serial was deprecated in QEMU 2.10. It's time to > remove it. > > Tests need to be updated to set the serial number with -global instead > of using the -drive option. > > Signed-off-by: Kevin Wolf > --- > include/hw/block/block.h | 1 - >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 01/10] qapi: Add default-variant for flat unions

2018-06-13 Thread Max Reitz
On 2018-06-12 17:25, Markus Armbruster wrote: > Max Reitz writes: > >> This patch allows specifying a discriminator that is an optional member >> of the base struct. In such a case, a default value must be provided >> that is used when no value is given. > > Hmm. Can you explain why you need

Re: [Qemu-block] [Qemu-devel] [PATCH 2/3] block: Remove deprecated -drive option addr

2018-06-13 Thread Markus Armbruster
Kevin Wolf writes: > The -drive option addr was deprecated in QEMU 2.10. It's time to remove > it. > > Signed-off-by: Kevin Wolf One of my first additions to QEMU... and a bit of an embarrassment. Reviewed-by: Markus Armbruster

Re: [Qemu-block] [Qemu-devel] [PATCH 1/3] block: Remove deprecated -drive geometry options

2018-06-13 Thread Markus Armbruster
Kevin Wolf writes: > The -drive options cyls, heads, secs and trans were deprecated in > QEMU 2.10. It's time to remove them. > > hd-geo-test tested both the old version with geometry options in -drive > and the new one with -device. Therefore the code using -drive doesn't > have to be replaced

[Qemu-block] [PATCH 2/3] block: Remove deprecated -drive option addr

2018-06-13 Thread Kevin Wolf
The -drive option addr was deprecated in QEMU 2.10. It's time to remove it. Signed-off-by: Kevin Wolf --- include/sysemu/blockdev.h | 1 - blockdev.c| 17 + device-hotplug.c | 4 qemu-doc.texi | 5 - qemu-options.hx | 5

[Qemu-block] [PATCH 0/3] block: Remove deprecated -drive options

2018-06-13 Thread Kevin Wolf
We deprecated a bunch of -drive options in 2.10, so let's remove them for 3.0. Kevin Wolf (3): block: Remove deprecated -drive geometry options block: Remove deprecated -drive option addr block: Remove deprecated -drive option serial include/hw/block/block.h | 1 -

[Qemu-block] [PATCH 3/3] block: Remove deprecated -drive option serial

2018-06-13 Thread Kevin Wolf
The -drive option serial was deprecated in QEMU 2.10. It's time to remove it. Tests need to be updated to set the serial number with -global instead of using the -drive option. Signed-off-by: Kevin Wolf --- include/hw/block/block.h | 1 - include/sysemu/blockdev.h | 1 -

[Qemu-block] [PATCH 1/3] block: Remove deprecated -drive geometry options

2018-06-13 Thread Kevin Wolf
The -drive options cyls, heads, secs and trans were deprecated in QEMU 2.10. It's time to remove them. hd-geo-test tested both the old version with geometry options in -drive and the new one with -device. Therefore the code using -drive doesn't have to be replaced there, we just need to remove

Re: [Qemu-block] [Qemu-devel] [PATCH v4 11/40] hw/xen: Use the IEC binary prefix definitions

2018-06-13 Thread Eric Blake
On 06/12/2018 04:10 PM, Richard Henderson wrote: So there's tradeoffs either way, and you at least need to document in your commit messages what auditing you have done that any type changes introduced by your changes are safe. I'm more concerned about unnecessary or unintended signed vs

Re: [Qemu-block] [PATCH 1/2] xen: add xen disk naming for use in monitor

2018-06-13 Thread Anthony PERARD
On Tue, Jun 12, 2018 at 05:51:02PM -0600, Bruce Rogers wrote: > Provide monitor naming of xen disks, including associating an > attached dev_id for a BlockBackend which has legacy_dev set. > Currently, only xen disks have legacy_dev set to true. > > Signed-off-by: Bruce Rogers > --- >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] qapi: query-blockstat: add driver specific file-posix stats

2018-06-13 Thread Markus Armbruster
Anton Nefedov writes: > On 8/6/2018 8:29 AM, Markus Armbruster wrote: >> Eric Blake writes: >> >>> On 06/07/2018 10:23 AM, Anton Nefedov wrote: >> If we introduce BlockdevDriver as a discriminator as Markus suggests >> above, we need some way to define its value. >> I guess one

Re: [Qemu-block] [Qemu-devel] [PATCH v2 8/8] qapi: query-blockstat: add driver specific file-posix stats

2018-06-13 Thread Anton Nefedov
On 8/6/2018 8:29 AM, Markus Armbruster wrote: Eric Blake writes: On 06/07/2018 10:23 AM, Anton Nefedov wrote: If we introduce BlockdevDriver as a discriminator as Markus suggests above, we need some way to define its value. I guess one would be to check blk->bs->drv->format_name but it

Re: [Qemu-block] [PATCH] nvme: Support image creation

2018-06-13 Thread Fam Zheng
On Wed, 06/13 10:06, Kevin Wolf wrote: > Am 13.06.2018 um 09:46 hat Fam Zheng geschrieben: > > Similar to the host_device's implementation, we check the requested > > length against the namespace size. > > > > Truncation is necessary to make qcow2 creation work. > > > > Signed-off-by: Fam Zheng

Re: [Qemu-block] [PATCH] nvme: Reset s->nr_queues upon open failure

2018-06-13 Thread Kevin Wolf
Am 13.06.2018 um 09:45 hat Fam Zheng geschrieben: > It is wrong to leave this field as 1, as nvme_close() called in the > error handling code in nvme_file_open() will use it and try to free > s->queues again. > > Clear the fields to avoid double-free. > > Cc: qemu-sta...@nongnu.org >

Re: [Qemu-block] [PATCH] nvme: Support image creation

2018-06-13 Thread Kevin Wolf
Am 13.06.2018 um 09:46 hat Fam Zheng geschrieben: > Similar to the host_device's implementation, we check the requested > length against the namespace size. > > Truncation is necessary to make qcow2 creation work. > > Signed-off-by: Fam Zheng > +static int coroutine_fn

[Qemu-block] [PATCH] nvme: Support image creation

2018-06-13 Thread Fam Zheng
Similar to the host_device's implementation, we check the requested length against the namespace size. Truncation is necessary to make qcow2 creation work. Signed-off-by: Fam Zheng --- block/nvme.c | 72 1 file changed, 72 insertions(+)

[Qemu-block] [PATCH] nvme: Reset s->nr_queues upon open failure

2018-06-13 Thread Fam Zheng
It is wrong to leave this field as 1, as nvme_close() called in the error handling code in nvme_file_open() will use it and try to free s->queues again. Clear the fields to avoid double-free. Cc: qemu-sta...@nongnu.org Signed-off-by: Fam Zheng --- block/nvme.c | 2 ++ 1 file changed, 2

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/4] vmdk: Implement .bdrv_co_create callback

2018-06-13 Thread Markus Armbruster
Still only looking at QAPI-related aspects. Fam Zheng writes: > This makes VMDK support x-blockdev-create. The implementation reuses the > image creation code in vmdk_co_create_opts which now acceptes a callback > pointer to "retrieve" BlockBackend pointers from the caller. This way we >