Re: [Qemu-block] [PATCH v12 00/10] qcow2 zero-cluster tweaks [was add blkdebug tests]

On 05/05/2017 05:18 PM, Max Reitz wrote: > On 04.05.2017 05:07, Eric Blake wrote: >> I've collected several improvements for qcow2 zero-cluster handling. >> >> Available as a tag at: >> git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v12 >> >> Marked as v12 for "hysterical raisins", since

Re: [Qemu-block] [PATCH v12 08/10] qcow2: Optimize write zero of unaligned tail cluster

On 05/05/2017 05:06 PM, Max Reitz wrote: > On 04.05.2017 05:07, Eric Blake wrote: >> We've already improved discards to operate efficiently on the tail >> of an unaligned qcow2 image; it's time to make a similar improvement >> to write zeroes. The special case is only valid at the tail >> cluster

Re: [Qemu-block] [PATCH v11 9/9] tests: Add coverage for recent block geometry fixes

On 29.04.2017 21:14, Eric Blake wrote: > Use blkdebug's new geometry constraints to emulate setups that > have needed past regression fixes: write zeroes asserting > when running through a loopback block device with max-transfer > smaller than cluster size, and discard rounding away portions > of

Re: [Qemu-block] [PATCH v12 00/10] qcow2 zero-cluster tweaks [was add blkdebug tests]

On 04.05.2017 05:07, Eric Blake wrote: > I've collected several improvements for qcow2 zero-cluster handling. > > Available as a tag at: > git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v12 > > Marked as v12 for "hysterical raisins", since it it the half of > v10 [1] that was not

Re: [Qemu-block] [PATCH v12 08/10] qcow2: Optimize write zero of unaligned tail cluster

On 04.05.2017 05:07, Eric Blake wrote: > We've already improved discards to operate efficiently on the tail > of an unaligned qcow2 image; it's time to make a similar improvement > to write zeroes. The special case is only valid at the tail > cluster of a file, where we must recognize that any

Re: [Qemu-block] [PATCH v12 06/10] iotests: Improve _filter_qemu_img_map

On 05.05.2017 23:06, Eric Blake wrote: > On 05/05/2017 03:58 PM, Max Reitz wrote: >> On 04.05.2017 05:07, Eric Blake wrote: >>> Although _filter_qemu_img_map documents that it scrubs offsets, it >>> was only doing so for human mode. Of the existing tests using the >>> filter (97, 122, 150, 154,

Re: [Qemu-block] [PATCH v12 06/10] iotests: Improve _filter_qemu_img_map

On 05/05/2017 03:58 PM, Max Reitz wrote: > On 04.05.2017 05:07, Eric Blake wrote: >> Although _filter_qemu_img_map documents that it scrubs offsets, it >> was only doing so for human mode. Of the existing tests using the >> filter (97, 122, 150, 154, 176), two of them are affected, but it >> does

Re: [Qemu-block] [PATCH v12 06/10] iotests: Improve _filter_qemu_img_map

On 04.05.2017 05:07, Eric Blake wrote: > Although _filter_qemu_img_map documents that it scrubs offsets, it > was only doing so for human mode. Of the existing tests using the > filter (97, 122, 150, 154, 176), two of them are affected, but it > does not hurt the validity of the tests to not

Re: [Qemu-block] [PATCH v12 04/10] qcow2: Make distinction between zero cluster types obvious

On 04.05.2017 05:07, Eric Blake wrote: > Treat plain zero clusters differently from allocated ones, so that > we can simplify the logic of checking whether an offset is present. > Do this by splitting QCOW2_CLUSTER_ZERO into two new enums, > QCOW2_CLUSTER_ZERO_PLAIN and QCOW2_CLUSTER_ZERO_ALLOC. >

Re: [Qemu-block] [PATCH v12 05/10] qcow2: Optimize zero_single_l2() to minimize L2 churn

On 04.05.2017 05:07, Eric Blake wrote: > Similar to discard_single_l2(), we should try to avoid dirtying > the L2 cache when the cluster we are changing already has the > right characteristics. > > Note that by the time we get to zero_single_l2(), BDRV_REQ_MAY_UNMAP > is a requirement to

[Qemu-block] [PATCH] fixup! sysbus: Set user_creatable=false by default on TYPE_SYS_BUS_DEVICE

On Fri, May 05, 2017 at 01:54:19PM -0300, Eduardo Habkost wrote: > On Fri, May 05, 2017 at 09:36:22AM +0200, Cornelia Huck wrote: > [...] > > > diff --git a/hw/core/sysbus.c b/hw/core/sysbus.c > > > index c0f560b289..6a2eec8dd0 100644 > > > --- a/hw/core/sysbus.c > > > +++ b/hw/core/sysbus.c > >

Re: [Qemu-block] [PATCH v12 03/10] qcow2: Correctly report status of preallocated zero clusters

On 04.05.2017 05:07, Eric Blake wrote: > We were throwing away the preallocation information associated with > zero clusters. But we should be matching the well-defined semantics > in bdrv_get_block_status(), where (BDRV_BLOCK_ZERO | > BDRV_BLOCK_OFFSET_VALID) informs the user which offset is

Re: [Qemu-block] [PATCH v12 02/10] block: Update comments on BDRV_BLOCK_* meanings

On 04.05.2017 05:07, Eric Blake wrote: > We had some conflicting documentation: a nice 8-way table that > described all possible combinations of DATA, ZERO, and > OFFSET_VALID, contrasted with text that implied that OFFSET_VALID > always meant raw data could be read directly. Furthermore, the >

Re: [Qemu-block] [PATCH v12 01/10] qcow2: Use consistent switch indentation

On 04.05.2017 05:07, Eric Blake wrote: > Fix a couple of inconsistent indentations, before an upcoming > patch further tweaks the switch statements. While at it, make > some tweaks for shorter lines to keep checkpatch happy (best > viewed with 'git diff -b'). > > Signed-off-by: Eric Blake

[Qemu-block] [PATCH v6 5/5] shutdown: Expose bool cause in SHUTDOWN and RESET events

Libvirt would like to be able to distinguish between a SHUTDOWN event triggered solely by guest request and one triggered by a SIGTERM or other action on the host. While qemu_kill_report() was already able to give different output to stderr based on whether a shutdown was triggered by a host

Re: [Qemu-block] [PATCH] block: Simplify BDRV_BLOCK_RAW recursion

On 04.05.2017 19:37, Eric Blake wrote: > Since we are already in coroutine context during the body of > bdrv_co_get_block_status(), we can shave off a few layers of > wrappers when recursing to query the protocol when a format driver > returned BDRV_BLOCK_RAW. > > Note that we are already using

Re: [Qemu-block] [PATCH RESEND v2 02/21] sysbus: Set user_creatable=false by default on TYPE_SYS_BUS_DEVICE

On Fri, May 05, 2017 at 09:36:22AM +0200, Cornelia Huck wrote: [...] > > diff --git a/hw/core/sysbus.c b/hw/core/sysbus.c > > index c0f560b289..6a2eec8dd0 100644 > > --- a/hw/core/sysbus.c > > +++ b/hw/core/sysbus.c > > @@ -326,6 +326,17 @@ static void sysbus_device_class_init(ObjectClass > >

Re: [Qemu-block] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Thu, Apr 27, 2017 at 01:36:30PM +0530, Ashijeet Acharya wrote: > This series helps to provide chunk size independence for DMG driver to prevent > denial-of-service in cases where untrusted files are being accessed by the > user. The core of the chunk size dependence problem are these lines:

Re: [Qemu-block] [PATCH v2 3/7] dmg: Refactor and prepare dmg_read_chunk() to cache random access points

On Thu, Apr 27, 2017 at 01:36:33PM +0530, Ashijeet Acharya wrote: > Refactor dmg_read_chunk() to start making use of the new DMGReadState > structure and do chunk and sector related calculations based on it. > Add a new argument "DMGReadState *drs" to it. > > Also, rename DMG_SECTORCOUNTS_MAX to

Re: [Qemu-block] [PATCH v2 0/7] Refactor DMG driver to have chunk size independence

On Thu, Apr 27, 2017 at 01:36:30PM +0530, Ashijeet Acharya wrote: > ->Testing procedure: > Convert a DMG file to raw format using the "qemu-img convert" tool present in > v2.9.0 > Next convert the same image again after applying these patches. > Compare the two images using "qemu-img compare" tool

Re: [Qemu-block] [PATCH for 2.10 00/17] Block layer thread safety, part 1

On Thu, Apr 20, 2017 at 02:00:41PM +0200, Paolo Bonzini wrote: > This series uses mutexes or atomic operations around core block layer > operations. The remaining parts include: > > - drivers, though most of them are already thread safe (part 2, 8 patches, > depends on Kevin's conversion of

Re: [Qemu-block] [PATCH 17/17] block: make accounting thread-safe

On Thu, Apr 20, 2017 at 02:00:58PM +0200, Paolo Bonzini wrote: > I'm not trying too hard yet. Later, with multiqueue support, > this may cause cacheline bouncing. This patch calls qemu_clock_get_ns() within a spinlock region. It's probably just a memory load, rdtsc, and some math but more

Re: [Qemu-block] [PATCH 03/18] nbd: Minimal structured read for server

04.05.2017 16:28, Eric Blake wrote: On 05/04/2017 05:58 AM, Vladimir Sementsov-Ogievskiy wrote: @@ -70,6 +70,25 @@ struct NBDSimpleReply { }; typedef struct NBDSimpleReply NBDSimpleReply; +typedef struct NBDStructuredReplyChunk { +uint32_t magic; +uint16_t flags; +uint16_t

Re: [Qemu-block] [PATCH 14/17] block: optimize access to reqs_lock

On 05/05/2017 12:25, Stefan Hajnoczi wrote: > On Thu, May 04, 2017 at 06:06:39PM +0200, Paolo Bonzini wrote: >> On 04/05/2017 16:59, Stefan Hajnoczi wrote: >>> On Thu, Apr 20, 2017 at 02:00:55PM +0200, Paolo Bonzini wrote: Hot path reqs_lock critical sections are very small; the only large

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: 147: Don't test inet6 if not available

On Fri, May 05, 2017 at 06:21:53PM +0800, Fam Zheng wrote: > This is the case in our docker tests, as we use --net=none there. Skip > this method. > > Signed-off-by: Fam Zheng > --- > tests/qemu-iotests/147 | 7 +++ > 1 file changed, 7 insertions(+) > > diff --git

Re: [Qemu-block] [PATCH 16/17] block: protect modification of dirty bitmaps with a mutex

On 05/05/2017 12:36, Stefan Hajnoczi wrote: > On Thu, Apr 20, 2017 at 02:00:57PM +0200, Paolo Bonzini wrote: >> @@ -410,6 +442,18 @@ int bdrv_get_dirty(BlockDriverState *bs, >> BdrvDirtyBitmap *bitmap, >> } >> } >> >> +int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, >>

Re: [Qemu-block] [PATCH 16/17] block: protect modification of dirty bitmaps with a mutex

On Thu, Apr 20, 2017 at 02:00:57PM +0200, Paolo Bonzini wrote: > @@ -410,6 +442,18 @@ int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap > *bitmap, > } > } > > +int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap, > + int64_t sector) Is it a good idea

[Qemu-block] [PATCH] iotests: 147: Don't test inet6 if not available

This is the case in our docker tests, as we use --net=none there. Skip this method. Signed-off-by: Fam Zheng --- tests/qemu-iotests/147 | 7 +++ 1 file changed, 7 insertions(+) diff --git a/tests/qemu-iotests/147 b/tests/qemu-iotests/147 index 32afea6..db34838 100755 ---

Re: [Qemu-block] [PATCH 14/17] block: optimize access to reqs_lock

On Thu, May 04, 2017 at 06:06:39PM +0200, Paolo Bonzini wrote: > On 04/05/2017 16:59, Stefan Hajnoczi wrote: > > On Thu, Apr 20, 2017 at 02:00:55PM +0200, Paolo Bonzini wrote: > >> Hot path reqs_lock critical sections are very small; the only large > >> critical > >> sections happen when a

Re: [Qemu-block] [PATCH v2] aio: add missing aio_notify() to aio_enable_external()

On Fri, 05/05 11:02, Stefan Hajnoczi wrote: > diff --git a/include/block/aio.h b/include/block/aio.h > index 406e323..b3d6f76 100644 > --- a/include/block/aio.h > +++ b/include/block/aio.h > @@ -454,8 +454,14 @@ static inline void aio_disable_external(AioContext *ctx) > */ > static inline void

Re: [Qemu-block] [PATCH] nvme: Implement Write Zeroes

On Fri, May 05, 2017 at 12:03:40PM +0200, Paolo Bonzini wrote: > While that's allowed and it makes sense indeed on SSDs, for QEMU's > typical usage it can lead to fragmentation and worse performance. On > extent-based file systems, write zeroes without deallocate can be > implemented very

Re: [Qemu-block] [PATCH] nvme: Implement Write Zeroes

On 05/05/2017 11:51, Christoph Hellwig wrote: >> could you pass BDRV_REQ_MAY_UNMAP for the flags here if the deallocate >> bit (dword 12 bit 25) is set? > In fact we should do that unconditonally. The deallocate bit is new > in 1.3 (which we don't claim to support) and forces deallocating, but

Re: [Qemu-block] [PATCH] aio: add missing aio_notify() to aio_enable_external()

On Thu, May 04, 2017 at 06:54:58PM +0800, Fam Zheng wrote: > On Thu, 05/04 12:36, Paolo Bonzini wrote: > > On 04/05/2017 12:23, Stefan Hajnoczi wrote: > > > diff --git a/include/block/aio.h b/include/block/aio.h > > > index 406e323..5294b04 100644 > > > --- a/include/block/aio.h > > > +++

[Qemu-block] [PATCH v2] aio: add missing aio_notify() to aio_enable_external()

The main loop uses aio_disable_external()/aio_enable_external() to temporarily disable processing of external AioContext clients like device emulation. This allows monitor commands to quiesce I/O and prevent the guest from submitting new requests while a monitor command is in progress. The

[Qemu-block] [PATCH v2] nvme: Implement Write Zeroes

Signed-off-by: Keith Busch [hch: ported over from qemu-nvme.git to mainline] Signed-off-by: Christoph Hellwig --- hw/block/nvme.c | 26 ++ hw/block/nvme.h | 1 + 2 files changed, 27 insertions(+) Changes since v1: - add

Re: [Qemu-block] [PATCH] nvme: Implement Write Zeroes

On Fri, May 05, 2017 at 11:30:11AM +0200, Paolo Bonzini wrote: > could you pass BDRV_REQ_MAY_UNMAP for the flags here if the deallocate > bit (dword 12 bit 25) is set? In fact we should do that unconditonally. The deallocate bit is new in 1.3 (which we don't claim to support) and forces

Re: [Qemu-block] [PATCH] nvme: Implement Write Zeroes

On 05/05/2017 11:00, Christoph Hellwig wrote: > Signed-off-by: Keith Busch > [hch: ported over from qemu-nvme.git to mainline] > Signed-off-by: Christoph Hellwig > --- > hw/block/nvme.c | 26 ++ > hw/block/nvme.h | 1 + > 2 files

[Qemu-block] [PATCH] nvme: Implement Write Zeroes

Signed-off-by: Keith Busch [hch: ported over from qemu-nvme.git to mainline] Signed-off-by: Christoph Hellwig --- hw/block/nvme.c | 26 ++ hw/block/nvme.h | 1 + 2 files changed, 27 insertions(+) diff --git a/hw/block/nvme.c

[Qemu-block] [PATCH v4] migration/block: move bdrv_is_allocated() into bb's AioContext

From: Lidong Chen when block migration with high-speed, mig_save_device_bulk hold the BQL and invoke bdrv_is_allocated frequently. This patch moves bdrv_is_allocated() into bb's AioContext. It will execute without blocking other I/O activity. Signed-off-by: Lidong Chen

Re: [Qemu-block] [PATCH RESEND v2 02/21] sysbus: Set user_creatable=false by default on TYPE_SYS_BUS_DEVICE

On Wed, 3 May 2017 17:35:45 -0300 Eduardo Habkost wrote: > commit 33cd52b5d7b9adfd009e95f07e6c64dd88ae2a31 unset > cannot_instantiate_with_device_add_yet in TYPE_SYSBUS, making all > sysbus devices appear on "-device help" and lack the "no-user" > flag in "info qdm". > >

Re: [Qemu-block] [PATCH] MAINTAINERS: Update paths for AioContext implementation

Applied to -trivial, mentioning c2b38b2 in commit message, thanks! /mjt