Re: [Qemu-block] [PATCH] virtio: make features 64bit wide

2015-05-29 Thread Gerd Hoffmann
On Fr, 2015-05-29 at 09:48 +0200, Gerd Hoffmann wrote: Make features 64bit wide everywhere. Exception: command line flags remain 32bit and are copyed into the lower 32 host_features at initialization time. Oops, forgot to git commit --amend, scratch this, v2 coming. sorry, Gerd

Re: [Qemu-block] [PATCH 2/3] qcow2: add option to clean unused cache entries after some time

2015-05-29 Thread Kevin Wolf
Am 28.05.2015 um 21:41 hat Eric Blake geschrieben: On 05/28/2015 09:23 AM, Max Reitz wrote: Can we mark the parameter optional, and only provide it when it is non-zero? That way, qemu-img (which cannot set an interval) will not report it, and the only time it will appear is if it was set

[Qemu-block] [PATCH] virtio: make features 64bit wide

2015-05-29 Thread Gerd Hoffmann
Make features 64bit wide everywhere. Exception: command line flags remain 32bit and are copyed into the lower 32 host_features at initialization time. On migration a full 64bit guest_features field is sent if one of the high bits is set, additionally to the lower 32bit guest_features field which

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Dr. David Alan Gilbert
* Markus Armbruster (arm...@redhat.com) wrote: Dr. David Alan Gilbert dgilb...@redhat.com writes: * John Snow (js...@redhat.com) wrote: On 05/21/2015 09:19 AM, Kevin Wolf wrote: The floppy controller spec describes three different controller phases, which are currently not

[Qemu-block] [PATCH v2] virtio: make features 64bit wide

2015-05-29 Thread Gerd Hoffmann
Make features 64bit wide everywhere. Exception: command line flags remain 32bit and are copyed into the lower 32 host_features at initialization time. On migration a full 64bit guest_features field is sent if one of the high bits is set, additionally to the lower 32bit guest_features field which

Re: [Qemu-block] [RFC PATCH 01/12] block: Use bdrv_drain to replace uncessary bdrv_drain_all

2015-05-29 Thread Paolo Bonzini
On 29/05/2015 12:53, Fam Zheng wrote: There callers work on a single BlockDriverState subtree, where using bdrv_drain() is more accurate. Signed-off-by: Fam Zheng f...@redhat.com --- block.c | 6 +++--- block/snapshot.c | 2 +- migration/block.c | 2 +- 3 files changed, 5

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/4] tighten conditions for board-implied FDC in pc-q35-2.4+

2015-05-29 Thread Markus Armbruster
Laszlo Ersek ler...@redhat.com writes: Version 2 of http://lists.nongnu.org/archive/html/qemu-devel/2015-05/msg05640.html. Changes are broken out per-patch; the cumulative changes are: - more granular structure (several patches in place of 1), - rename force_fdctrl parameter to

Re: [Qemu-block] [RFC PATCH 03/12] blockdev: Lock BDS during internal snapshot transaction

2015-05-29 Thread Paolo Bonzini
On 29/05/2015 12:53, Fam Zheng wrote: Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/blockdev.c b/blockdev.c index 5eaf77e..46f8d60 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1262,6 +1262,7 @@ typedef struct

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: * Markus Armbruster (arm...@redhat.com) wrote: Dr. David Alan Gilbert dgilb...@redhat.com writes: * John Snow (js...@redhat.com) wrote: On 05/21/2015 09:19 AM, Kevin

[Qemu-block] [RFC PATCH 02/12] block: Introduce bdrv_lock and bdrv_unlock API

2015-05-29 Thread Fam Zheng
For various purposes, BDS users call bdrv_drain or bdrv_drain_all to make sure there are no pending requests duringA a series of operations on the BDS. But in the middle of operations, the caller may 1) yield from a coroutine (mirror_run); 2) defer the next part of work to a BH (mirror_run); 3)

[Qemu-block] [RFC PATCH 01/12] block: Use bdrv_drain to replace uncessary bdrv_drain_all

2015-05-29 Thread Fam Zheng
There callers work on a single BlockDriverState subtree, where using bdrv_drain() is more accurate. Signed-off-by: Fam Zheng f...@redhat.com --- block.c | 6 +++--- block/snapshot.c | 2 +- migration/block.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git

[Qemu-block] [RFC PATCH 03/12] blockdev: Lock BDS during internal snapshot transaction

2015-05-29 Thread Fam Zheng
Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/blockdev.c b/blockdev.c index 5eaf77e..46f8d60 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1262,6 +1262,7 @@ typedef struct InternalSnapshotState { BlockDriverState *bs;

[Qemu-block] [RFC PATCH 05/12] blockdev: Lock BDS during drive-backup transaction

2015-05-29 Thread Fam Zheng
To save the bs pointer for drive_backup_clean, pull the assignment to state-bs up. It will not be a problem for drive_backup_abort because state-job will still be NULL. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

[Qemu-block] [RFC PATCH 00/12] block: Protect block jobs with lock / unlock API

2015-05-29 Thread Fam Zheng
This is the partial work to introduce bdrv_lock / bdrv_unlock and use them in block jobs where exclusive access to a BDS is necessary. It address the same category of problems as [1] with a different API, as the idea proposed by Paolo and Kevin. What's implemented in this series is also very

[Qemu-block] [RFC PATCH 10/12] virtio-scsi-dataplane: Add backend lock listener

2015-05-29 Thread Fam Zheng
When a disk is attached to scsi-bus, virtio_scsi_hotplug will take care of protecting the block device with op blockers. Currently we haven't enabled block jobs (like what's done in virtio_blk_data_plane_create), but it is better to disable ioeventfd when backend is locked. This is useful to make

Re: [Qemu-block] [Qemu-devel] [PATCH v6 01/13] block: Add op blocker type device IO

2015-05-29 Thread Andrey Korolyov
On Thu, May 28, 2015 at 3:05 PM, Fam Zheng f...@redhat.com wrote: On Thu, 05/28 13:19, Paolo Bonzini wrote: On 28/05/2015 13:11, Fam Zheng wrote: Whoever uses ioeventfd needs to implement pause/resume, yes---not just dataplane, also regular virtio-blk/virtio-scsi. However, everyone

[Qemu-block] [PATCH 3/3] qcow2: reorder fields in Qcow2CachedTable to reduce padding

2015-05-29 Thread Alberto Garcia
Changing the current ordering saves 8 bytes per cache entry in x86_64. Signed-off-by: Alberto Garcia be...@igalia.com Reviewed-by: Max Reitz mre...@redhat.com Reviewed-by: Eric Blake ebl...@redhat.com --- block/qcow2-cache.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

[Qemu-block] [PATCH 1/3] qcow2: mark the memory as no longer needed after qcow2_cache_empty()

2015-05-29 Thread Alberto Garcia
After having emptied the cache, the data in the cache tables is no longer useful, so we can tell the kernel that we are done with it. In Linux this frees the resources associated with it. The effect of this can be seen in the HMP commit operation: it moves data from the top to the base image (and

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Kevin Wolf
Am 29.05.2015 um 11:38 hat Dr. David Alan Gilbert geschrieben: * Kevin Wolf (kw...@redhat.com) wrote: Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: * Markus Armbruster (arm...@redhat.com) wrote: Dr. David Alan Gilbert dgilb...@redhat.com writes: * John Snow

[Qemu-block] [RFC PATCH 04/12] blockdev: Lock BDS during external snapshot transaction

2015-05-29 Thread Fam Zheng
Lock immediately follows aio_context_acquire, so unlock right before the corresponding aio_context_release. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/blockdev.c b/blockdev.c index 46f8d60..1c3946a 100644 --- a/blockdev.c +++

[Qemu-block] [RFC PATCH 06/12] blockdev: Lock BDS during blockdev-backup transaction

2015-05-29 Thread Fam Zheng
The change is similar to drive-backup in last patch. Signed-off-by: Fam Zheng f...@redhat.com --- blockdev.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/blockdev.c b/blockdev.c index 517def4..80cf7d5 100644 --- a/blockdev.c +++ b/blockdev.c @@ -1669,6 +1669,8 @@ static

[Qemu-block] [RFC PATCH 08/12] virtio-blk: Move complete_request to 'ops' structure

2015-05-29 Thread Fam Zheng
Should more ops be added to differentiate code between dataplane and non-dataplane, the new saved_ops approach will be cleaner than messing with N pointers. Signed-off-by: Fam Zheng f...@redhat.com Reviewed-by: Max Reitz mre...@redhat.com --- hw/block/dataplane/virtio-blk.c | 13 -

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Kevin Wolf
Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: * Markus Armbruster (arm...@redhat.com) wrote: Dr. David Alan Gilbert dgilb...@redhat.com writes: * John Snow (js...@redhat.com) wrote: On 05/21/2015 09:19 AM, Kevin Wolf wrote: The floppy controller spec

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Dr. David Alan Gilbert
* Peter Maydell (peter.mayd...@linaro.org) wrote: On 29 May 2015 at 11:34, Dr. David Alan Gilbert dgilb...@redhat.com wrote: It's the destination I'm worried about here, not the source; lets say you have two devices, a b. 'a' gets serialised, but then 'b' finds it has to wait, so we

[Qemu-block] [PATCH 2/3] qcow2: add option to clean unused cache entries after some time

2015-05-29 Thread Alberto Garcia
This adds a new 'cache-clean-interval' option that cleans all qcow2 cache entries that haven't been used in a certain interval, given in seconds. This allows setting a large L2 cache size so it can handle scenarios with lots of I/O and at the same time use little memory during periods of

[Qemu-block] [PATCH v4 0/3] Clean unused entries in the qcow2 L2/refcount cache

2015-05-29 Thread Alberto Garcia
v4: - Revert the 'cache-clean-interval' change. This should probably go into a new BlockDeviceInfoSpecific struct (along with other settings), but is out of the scope for this series. v3: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg05473.html - Add 'cache-clean-interval' field to

[Qemu-block] [RFC PATCH 07/12] block-backend: Add blk_add_lock_unlock_notifier

2015-05-29 Thread Fam Zheng
Forward the call to bdrv_add_lock_unlock_notifier. Signed-off-by: Fam Zheng f...@redhat.com --- block/block-backend.c | 6 ++ include/sysemu/block-backend.h | 1 + 2 files changed, 7 insertions(+) diff --git a/block/block-backend.c b/block/block-backend.c index 93e46f3..79fdf75

[Qemu-block] [RFC PATCH 09/12] virtio-blk: Don't handle output when backend is locked

2015-05-29 Thread Fam Zheng
virtio-blk now listens to locking and unlocking of the associated block backend. Up on locking: non-dataplane: 1) Set VirtIOBlock.paused 2) In virtio_blk_handle_output, do nothing if VirtIOBlock.paused dataplane: 1) Clear the host event notifier 2) In handle_notify, do nothing

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Dr. David Alan Gilbert
* Kevin Wolf (kw...@redhat.com) wrote: Am 29.05.2015 um 11:38 hat Dr. David Alan Gilbert geschrieben: * Kevin Wolf (kw...@redhat.com) wrote: Am 29.05.2015 um 10:33 hat Dr. David Alan Gilbert geschrieben: * Markus Armbruster (arm...@redhat.com) wrote: Dr. David Alan Gilbert

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/8] fdc: Introduce fdctrl-phase

2015-05-29 Thread Peter Maydell
On 29 May 2015 at 11:34, Dr. David Alan Gilbert dgilb...@redhat.com wrote: It's the destination I'm worried about here, not the source; lets say you have two devices, a b. 'a' gets serialised, but then 'b' finds it has to wait, so we return to running the source and sending pages across.

[Qemu-block] [RFC PATCH 12/12] mirror: Protect source between bdrv_drain and bdrv_swap

2015-05-29 Thread Fam Zheng
Source and target are in sync when we leave the mirror_run loop, they should remain so until bdrv_swap. Before block_job_defer_to_main_loop was introduced, it has been easy to prove that. Now that tricky things can happen after mirror_run returns and before mirror_exit runs, for example, ioeventfd

Re: [Qemu-block] [Qemu-devel] [PATCH 8/8] qemu-iotests: expand test 093 to support group throttling

2015-05-29 Thread Stefan Hajnoczi
On Tue, May 19, 2015 at 03:24:36PM +0300, Alberto Garcia wrote: @@ -32,26 +34,31 @@ class ThrottleTestCase(iotests.QMPTestCase): raise Exception(Device not found for blockstats: %s % device) def setUp(self): -self.vm = iotests.VM().add_drive(self.test_img) +

Re: [Qemu-block] [Qemu-devel] [PATCH 7/8] throttle: Update throttle infrastructure copyright

2015-05-29 Thread Stefan Hajnoczi
On Tue, May 19, 2015 at 03:24:35PM +0300, Alberto Garcia wrote: Signed-off-by: Alberto Garcia be...@igalia.com --- include/qemu/throttle.h | 8 +--- util/throttle.c | 8 +--- 2 files changed, 10 insertions(+), 6 deletions(-) Reviewed-by: Stefan Hajnoczi stefa...@redhat.com

Re: [Qemu-block] [Qemu-devel] [PATCH 5/8] throttle: acquire the ThrottleGroup lock in bdrv_swap()

2015-05-29 Thread Stefan Hajnoczi
On Tue, May 19, 2015 at 03:24:33PM +0300, Alberto Garcia wrote: bdrv_swap() touches the fields of a BlockDriverState that are protected by the ThrottleGroup lock. Although those fields end up in their original place, they are temporarily swapped in the process, so there's a chance that an

Re: [Qemu-block] [PATCH] blockdev: no need to drain in qmp_block_commit

2015-05-29 Thread Stefan Hajnoczi
On Thu, May 28, 2015 at 04:21:43PM +0200, Paolo Bonzini wrote: Draining is not necessary, I/O can happen as soon as the commit coroutine yields. Draining can be necessary before reopening the file for read/write, or while modifying the backing file chain, but that is done separately in

Re: [Qemu-block] [Qemu-devel] [PATCH 8/8] qemu-iotests: expand test 093 to support group throttling

2015-05-29 Thread Alberto Garcia
On Fri 29 May 2015 02:32:45 PM CEST, Stefan Hajnoczi stefa...@gmail.com wrote: def setUp(self): -self.vm = iotests.VM().add_drive(self.test_img) +self.vm = iotests.VM() +for i in range(0, self.max_drives): +self.vm.add_drive(self.test_img) Unique

Re: [Qemu-block] [Qemu-devel] [PATCH 6/8] throttle: add the name of the ThrottleGroup to BlockDeviceInfo

2015-05-29 Thread Stefan Hajnoczi
On Tue, May 19, 2015 at 03:24:34PM +0300, Alberto Garcia wrote: Signed-off-by: Alberto Garcia be...@igalia.com --- block/qapi.c | 3 +++ hmp.c| 6 -- qapi/block-core.json | 4 +++- 3 files changed, 10 insertions(+), 3 deletions(-) Reviewed-by: Stefan Hajnoczi

Re: [Qemu-block] [PATCH] mirror: Skip block_job_defer_to_main_loop if already in the main context

2015-05-29 Thread Stefan Hajnoczi
On Fri, May 29, 2015 at 10:22:13AM +0800, Fam Zheng wrote: mirror_exit does the replacing, which requires source and target to be in sync, unfortunately we can't guarantee that before we have a complete block pause mechanism. So for non-dataplane block jobs, let's do the old thing as pre

Re: [Qemu-block] [Qemu-devel] [RFC PATCH 02/12] block: Introduce bdrv_lock and bdrv_unlock API

2015-05-29 Thread Eric Blake
On 05/29/2015 04:53 AM, Fam Zheng wrote: For various purposes, BDS users call bdrv_drain or bdrv_drain_all to make sure there are no pending requests duringA a series of operations on the BDS. But in s/duringA/during/ the middle of operations, the caller may 1) yield from a coroutine

Re: [Qemu-block] [PATCH v2] virtio: make features 64bit wide

2015-05-29 Thread Michael S. Tsirkin
On Fri, May 29, 2015 at 09:51:20AM +0200, Gerd Hoffmann wrote: Make features 64bit wide everywhere. Exception: command line flags remain 32bit and are copyed into the lower 32 host_features at initialization time. On migration a full 64bit guest_features field is sent if one of the high

Re: [Qemu-block] [PATCH v5 0/2] block: enforce minimal 4096 alignment in qemu_blockalign

2015-05-29 Thread Paolo Bonzini
On 13/05/2015 18:46, Denis V. Lunev wrote: I agree with this. Kernel guys are aware and may be we will have the fix after a while... I have heard (not tested) that performance loss over multi-queue SSD is around 30%. I came up with this patch... can you test it with your test case (and old