[Qemu-block] [PATCH] ratelimit: don't align wait time with slices

2018-02-06 Thread Wolfgang Bumiller
It is possible for rate limited writes to keep overshooting a slice's quota by a tiny amount causing the slice-aligned waiting period to effectively halve the rate. Signed-off-by: Wolfgang Bumiller --- Copied the Ccs from the discussion thread, hope that's fine, as I also

Re: [Qemu-block] [Qemu-devel] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-02-06 Thread Markus Armbruster
"Dr. David Alan Gilbert" writes: > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: >> 26.01.2018 18:05, Dr. David Alan Gilbert wrote: >> > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: [...] >> > > most of commands, ported to hmp are done

Re: [Qemu-block] [PATCH v5 0/5] coroutine-lock: polymorphic CoQueue

2018-02-06 Thread Fam Zheng
On Sat, 02/03 10:39, Paolo Bonzini wrote: > There are cases in which a queued coroutine must be restarted from > non-coroutine context (with qemu_co_enter_next). In this cases, > qemu_co_enter_next also needs to be thread-safe, but it cannot use a > CoMutex and so cannot qemu_co_queue_wait. This

Re: [Qemu-block] [PATCH for-2.9-rc5 v4 2/2] block: Drain BH in bdrv_drained_begin

2018-02-06 Thread Fam Zheng
On Tue, Feb 6, 2018 at 11:32 PM, Kevin Wolf wrote: > Am 18.04.2017 um 16:30 hat Fam Zheng geschrieben: >> During block job completion, nothing is preventing >> block_job_defer_to_main_loop_bh from being called in a nested >> aio_poll(), which is a trouble, such as in this code

Re: [Qemu-block] [Qemu-devel] [PATCH v8 18/26] block: Add sgfnt_runtime_opts to BlockDriver

2018-02-06 Thread Eric Blake
On 02/05/2018 09:18 AM, Max Reitz wrote: This new field can be set by block drivers to list the runtime options they accept that may influence the contents of the respective BDS. As of a follow-up patch, this list will be used by the common bdrv_refresh_filename() implementation to decide which

Re: [Qemu-block] [PATCH] block: Simplify bdrv_can_write_zeroes_with_unmap()

2018-02-06 Thread Eric Blake
On 01/29/2018 05:08 AM, Stefan Hajnoczi wrote: On Fri, Jan 26, 2018 at 01:34:39PM -0600, Eric Blake wrote: We don't need the can_write_zeroes_with_unmap field in BlockDriverInfo, because it is redundant information with supported_zero_flags & BDRV_REQ_MAY_UNMAP. Note that BlockDriverInfo and

Re: [Qemu-block] [PATCH 2/2] qapi: add block latency histogram interface

2018-02-06 Thread Eric Blake
On 02/06/2018 12:06 PM, Vladimir Sementsov-Ogievskiy wrote: 06.02.2018 18:50, Eric Blake wrote: On 02/06/2018 08:07 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. The

Re: [Qemu-block] [PATCH v2] iotests: 205: support luks format

2018-02-06 Thread Eric Blake
On 02/06/2018 12:26 PM, Daniel P. Berrangé wrote: On Tue, Feb 06, 2018 at 09:25:07PM +0300, Vladimir Sementsov-Ogievskiy wrote: Support default luks options in VM.add_drive and in new library function qemu_img_create. Use it in 205 iotests. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-02-06 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 26.01.2018 18:05, Dr. David Alan Gilbert wrote: > > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > > > 17.01.2018 19:03, Eric Blake wrote: > > > > On 01/17/2018 09:51 AM, Vladimir Sementsov-Ogievskiy wrote: > >

[Qemu-block] [PATCH v2] iotests: 205: support luks format

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
Support default luks options in VM.add_drive and in new library function qemu_img_create. Use it in 205 iotests. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: use keysec0 and IMGKEYSECRET tests/qemu-iotests/205| 4 ++-- tests/qemu-iotests/iotests.py |

Re: [Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Daniel P . Berrangé
On Tue, Feb 06, 2018 at 08:57:38PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 06.02.2018 20:29, Daniel P. Berrangé wrote: > > On Tue, Feb 06, 2018 at 08:16:42PM +0300, Vladimir Sementsov-Ogievskiy > > wrote: > > > Support default luks options in VM.add_drive and in new library > > > function

Re: [Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
06.02.2018 21:04, Eric Blake wrote: On 02/06/2018 11:57 AM, Vladimir Sementsov-Ogievskiy wrote: +luks_default_secret_id = 'luks_secret_default_iotests_id' Can we just call this  "keysec0", so we matchh convention used by the shell script based tests. Here I'm trying to avoid intersection

Re: [Qemu-block] [PATCH 2/2] qapi: add block latency histogram interface

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
06.02.2018 18:50, Eric Blake wrote: On 02/06/2018 08:07 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Eric Blake
On 02/06/2018 11:57 AM, Vladimir Sementsov-Ogievskiy wrote: +luks_default_secret_id = 'luks_secret_default_iotests_id' Can we just call this  "keysec0", so we matchh convention used by the shell script based tests. Here I'm trying to avoid intersection with some user-defined id. You're

Re: [Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
06.02.2018 20:29, Daniel P. Berrangé wrote: On Tue, Feb 06, 2018 at 08:16:42PM +0300, Vladimir Sementsov-Ogievskiy wrote: Support default luks options in VM.add_drive and in new library function qemu_img_create. Use it in 205 iotests. Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
06.02.2018 19:06, Eric Blake wrote: On 02/06/2018 09:29 AM, Vladimir Sementsov-Ogievskiy wrote: most of commands, ported to hmp are done in same style: they just call corresponding qmp command. Isn't it better to provide common interface for calling qmp commands through HMP monitor, to never

Re: [Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Daniel P . Berrangé
On Tue, Feb 06, 2018 at 08:16:42PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Support default luks options in VM.add_drive and in new library > function qemu_img_create. Use it in 205 iotests. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > instead of >

[Qemu-block] [PATCH] iotests: 205: support luks format

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
Support default luks options in VM.add_drive and in new library function qemu_img_create. Use it in 205 iotests. Signed-off-by: Vladimir Sementsov-Ogievskiy --- instead of [PATCH] iotests: 205: support only raw format let's just support luks. This patch also makes

Re: [Qemu-block] [PATCH] block: fix write with zero flag set and iovector provided

2018-02-06 Thread Stefan Hajnoczi
On Thu, Feb 01, 2018 at 05:16:31PM +0300, Anton Nefedov wrote: > The normal bdrv_co_pwritev() use is either > - BDRV_REQ_ZERO_WRITE reset and iovector provided > - BDRV_REQ_ZERO_WRITE set and iovector == NULL > > while > - the flag reset and iovector == NULL is an assertion failure > in

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-02-06 Thread Eric Blake
On 02/06/2018 09:29 AM, Vladimir Sementsov-Ogievskiy wrote: most of commands, ported to hmp are done in same style: they just call corresponding qmp command. Isn't it better to provide common interface for calling qmp commands through HMP monitor, to never create hmp versions of new commands?

Re: [Qemu-block] [PATCH 2/2] qapi: add block latency histogram interface

2018-02-06 Thread Eric Blake
On 02/06/2018 08:07 AM, Vladimir Sementsov-Ogievskiy wrote: Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 62

Re: [Qemu-block] [PATCH for-2.9-rc5 v4 2/2] block: Drain BH in bdrv_drained_begin

2018-02-06 Thread Kevin Wolf
Am 18.04.2017 um 16:30 hat Fam Zheng geschrieben: > During block job completion, nothing is preventing > block_job_defer_to_main_loop_bh from being called in a nested > aio_poll(), which is a trouble, such as in this code path: > > qmp_block_commit > commit_active_start >

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
26.01.2018 18:05, Dr. David Alan Gilbert wrote: * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: 17.01.2018 19:03, Eric Blake wrote: On 01/17/2018 09:51 AM, Vladimir Sementsov-Ogievskiy wrote: I have a script (for managing libvirt guest, but it can be adopted for qemu or even

Re: [Qemu-block] [Qemu-devel] [PATCH] block: include original filename when reporting invalid URIs

2018-02-06 Thread Eric Blake
On 02/06/2018 04:52 AM, Daniel P. Berrangé wrote: Consider passing a JSON based block driver to "qemu-img commit" $ qemu-img commit 'json:{"driver":"qcow2","file":{"driver":"gluster",\ "volume":"gv0","path":"sn1.qcow2", "server":[{"type":\

Re: [Qemu-block] [Qemu-devel] [PATCH v9 03/13] block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
24.01.2018 13:16, Paolo Bonzini wrote: On 22/01/2018 13:14, Vladimir Sementsov-Ogievskiy wrote: so, accessing the bitmap needs mutex lock? Then what do you mean under accessing the bitmap? Any touch of BdrvDirtyBitmap fields? Then "reading the list" will require bitmap mutex too. Or accessing

Re: [Qemu-block] [PATCH v8 21/26] block: Purify .bdrv_refresh_filename()

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:30 PM CET, Max Reitz wrote: > Currently, BlockDriver.bdrv_refresh_filename() is supposed to both > refresh the filename (BDS.exact_filename) and set BDS.full_open_options. > Now that we have generic code in the central bdrv_refresh_filename() for > creating

[Qemu-block] [PATCH 1/2] block/accounting: introduce latency histogram

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
Introduce latency histogram statics for block devices. For each accounted operation type latency region [0, +inf) is divided into subregions by several points. Then, calculate hits for each subregion. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

Re: [Qemu-block] [PATCH v8 25/26] block/curl: Implement bdrv_refresh_filename()

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:34 PM CET, Max Reitz wrote: > Signed-off-by: Max Reitz Reviewed-by: Alberto Garcia Berto

[Qemu-block] [PATCH 2/2] qapi: add block latency histogram interface

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
Set (and clear) histogram through new command block-latency-histogram-set and show new statistics in query-blockstats results. Signed-off-by: Vladimir Sementsov-Ogievskiy --- qapi/block-core.json | 62 +++- block/qapi.c

[Qemu-block] [PATCH 0/2] block latency histogram

2018-02-06 Thread Vladimir Sementsov-Ogievskiy
Vladimir Sementsov-Ogievskiy (2): block/accounting: introduce latency histogram qapi: add block latency histogram interface qapi/block-core.json | 62 +- include/block/accounting.h | 8 + block/accounting.c | 83

Re: [Qemu-block] [PATCH v8 24/26] block/curl: Harmonize option defaults

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:33 PM CET, Max Reitz wrote: > Both of the defaults we currently have in the curl driver are named > based on a slightly different schema, let's unify that and call both > CURL_BLOCK_OPT_${NAME}_DEFAULT. > > While at it, we can add a macro for the third option for which a

Re: [Qemu-block] [PATCH v8 20/26] block: Generically refresh runtime options

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:29 PM CET, Max Reitz wrote: > Instead of having every block driver which implements > bdrv_refresh_filename() copy all of the significant runtime options over > to bs->full_open_options, implement this process generically in > bdrv_refresh_filename(). > > This patch only

Re: [Qemu-block] [PATCH v8 05/26] block: Respect backing bs in bdrv_refresh_filename

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:14 PM CET, Max Reitz wrote: > Basically, bdrv_refresh_filename() should respect all children of a > BlockDriverState. However, generally those children are driver-specific, > so this function cannot handle the general case. On the other hand, > there are only few drivers

Re: [Qemu-block] [PATCH v8 04/26] iotests: Drop explicit base blockdev in 191

2018-02-06 Thread Alberto Garcia
On Mon 05 Feb 2018 04:18:13 PM CET, Max Reitz wrote: > Overriding the backing image should result in a json:{} pseudo-filename. > Then, you can no longer use the commit block job with filename > parameters. Therefore, do not explicitly add the base and override the > middle image in iotest 191,

Re: [Qemu-block] [PATCH v2 1/8] qapi: group BlockDeviceStats fields

2018-02-06 Thread Alberto Garcia
On Fri 19 Jan 2018 01:50:00 PM CET, Anton Nefedov wrote: > Make the stat fields definition slightly more readable. > Cosmetic change only. > > Signed-off-by: Anton Nefedov > --- > qapi/block-core.json | 24 ++-- > 1 file changed, 18 insertions(+),

Re: [Qemu-block] Block Migration and CPU throttling

2018-02-06 Thread Peter Lieven
Am 12.12.2017 um 18:05 schrieb Dr. David Alan Gilbert: * Peter Lieven (p...@kamp.de) wrote: Am 21.09.2017 um 14:36 schrieb Dr. David Alan Gilbert: * Peter Lieven (p...@kamp.de) wrote: Am 19.09.2017 um 16:41 schrieb Dr. David Alan Gilbert: * Peter Lieven (p...@kamp.de) wrote: Am 19.09.2017

[Qemu-block] [PATCH] block: include original filename when reporting invalid URIs

2018-02-06 Thread Daniel P . Berrangé
Consider passing a JSON based block driver to "qemu-img commit" $ qemu-img commit 'json:{"driver":"qcow2","file":{"driver":"gluster",\ "volume":"gv0","path":"sn1.qcow2", "server":[{"type":\ "tcp","host":"10.73.199.197","port":"24007"}]},}'