Re: [Qemu-block] [PATCH for-2.8] mirror: do not flush every time the disks are synced

- Original Message - > From: "Jeff Cody" > To: "Paolo Bonzini" > Cc: qemu-de...@nongnu.org, qemu-block@nongnu.org, js...@redhat.com > Sent: Wednesday, November 9, 2016 7:38:26 PM > Subject: Re: [PATCH for-2.8] mirror: do not flush every time the

Re: [Qemu-block] [PATCH for-2.8] block: Let write zeroes fallback work even with small max_transfer

On 11/09/2016 07:49 AM, Stefan Hajnoczi wrote: > On Tue, Nov 08, 2016 at 04:52:15PM -0600, Eric Blake wrote: >> Commit 443668ca rewrote the write_zeroes logic to guarantee that >> an unaligned request never crosses a cluster boundary. But >> in the rewrite, the new code assumed that at most one

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.8? 0/3] block/curl: Drop TFTP "support"

On Tue, Nov 08, 2016 at 08:14:58AM +0100, Markus Armbruster wrote: > Max Reitz writes: > > > On 07.11.2016 09:20, Markus Armbruster wrote: > >> Max Reitz writes: > >> > >>> On 03.11.2016 08:56, Markus Armbruster wrote: > Max Reitz

Re: [Qemu-block] [PATCH for-2.8] qemu-iotests: avoid spurious failure on test 109

On Wed, Nov 09, 2016 at 05:20:07PM +0100, Paolo Bonzini wrote: > In some cases it is possible that query-io-status is called just > before the job is completed, causing > > -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, > "event": "BLOCK_JOB_COMPLETED", "data":

Re: [Qemu-block] [PATCH for-2.8] mirror: do not flush every time the disks are synced

On Wed, Nov 09, 2016 at 05:20:08PM +0100, Paolo Bonzini wrote: > This puts a huge strain on the disks when there are many concurrent > migrations. With this patch we only flush twice: just before issuing > the event, and just before pivoting to the destination. If management > will complete the

Re: [Qemu-block] [PATCH for-2.8] qemu-iotests: avoid spurious failure on test 109

On Wed, Nov 09, 2016 at 05:20:07PM +0100, Paolo Bonzini wrote: > In some cases it is possible that query-io-status is called just > before the job is completed, causing > > -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, > "event": "BLOCK_JOB_COMPLETED", "data":

[Qemu-block] [PATCH 01/21] specs/qcow2: fix bitmap granularity qemu-specific note

Signed-off-by: Vladimir Sementsov-Ogievskiy --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0..dda53dd 100644 --- a/docs/specs/qcow2.txt +++ b/docs/specs/qcow2.txt @@

[Qemu-block] [PATCH 04/21] tests: add hbitmap iter test

Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- tests/test-hbitmap.c | 19 +++ 1 file changed, 19

[Qemu-block] [PATCH 03/21] hbitmap: improve dirty iter

Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/qemu/hbitmap.h | 25 +++-- util/hbitmap.c | 23 ++-

[Qemu-block] [PATCH 19/21] iotests: test qcow2 persistent dirty bitmap

Signed-off-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/165 | 89 ++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 95 insertions(+) create mode 100755

[Qemu-block] [PATCH 13/21] qcow2: add .bdrv_store_persistent_dirty_bitmaps()

Realize block bitmap storing interface, to allow qcow2 images store persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 451 +++ block/qcow2.c| 1 + block/qcow2.h|

[Qemu-block] [PATCH 14/21] block: add bdrv_can_store_dirty_bitmap

This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 22 ++

[Qemu-block] [PATCH 11/21] block: introduce persistent dirty bitmaps

New field BdrvDirtyBitmap.persistent means, that bitmap should be saved on bdrv_close, using format driver. Format driver should maintain bitmap storing. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 31 +++

[Qemu-block] [PATCH 20/21] qcow2-refcount: rename inc_refcounts() and make it public

This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-refcount.c | 53 ++ block/qcow2.h | 4 2 files

[Qemu-block] [PATCH 17/21] qmp: add autoload parameter to block-dirty-bitmap-add

Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- blockdev.c| 18 -- docs/qmp-commands.txt | 4 qapi/block-core.json | 6 +- 3 files changed, 25

[Qemu-block] [PATCH 08/21] block: introduce auto-loading bitmaps

Auto loading bitmaps are bitmaps store in the disk image, which should be loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block.c | 14 ++

[Qemu-block] [PATCH 06/21] block/dirty-bitmap: add deserialize_ones func

Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Reviewed-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c

[Qemu-block] [PATCH 09/21] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH 10/21] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 16

[Qemu-block] [PATCH 21/21] qcow2-bitmap: refcounts

Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img check implementation. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 73 ++ block/qcow2-refcount.c | 6 +

[Qemu-block] [PATCH 07/21] qcow2: add dirty bitmaps extension

Add dirty bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH 16/21] qmp: add persistent flag to block-dirty-bitmap-add

Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev --- blockdev.c| 17 - docs/qmp-commands.txt | 3 +++

[Qemu-block] [PATCH 18/21] qmp: add x-debug-block-dirty-bitmap-sha256

Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 5 + blockdev.c | 33 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 qapi/block-core.json |

[Qemu-block] [PATCH 12/21] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index

Re: [Qemu-block] [PATCH v4 0/6] jobs: fix transactional race condition

On Tue, Nov 08, 2016 at 01:50:33AM -0500, John Snow wrote: > There are a few problems with transactional job completion right now. > > First, if jobs complete so quickly they complete before remaining jobs > get a chance to join the transaction, the completion mode can leave well > known state

[Qemu-block] [PATCH for-2.8] mirror: do not flush every time the disks are synced

This puts a huge strain on the disks when there are many concurrent migrations. With this patch we only flush twice: just before issuing the event, and just before pivoting to the destination. If management will complete the job close to the BLOCK_JOB_READY event, the cost of the second flush

[Qemu-block] [PATCH for-2.8] qemu-iotests: avoid spurious failure on test 109

In some cases it is possible that query-io-status is called just before the job is completed, causing -{"timestamp": {"seconds": TIMESTAMP, "microseconds": TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 31457280, "offset": OFFSET, "speed": 0, "type":

Re: [Qemu-block] [PATCH v4 0/6] jobs: fix transactional race condition

On Tue, Nov 08, 2016 at 01:50:33AM -0500, John Snow wrote: > There are a few problems with transactional job completion right now. > > First, if jobs complete so quickly they complete before remaining jobs > get a chance to join the transaction, the completion mode can leave well > known state

Re: [Qemu-block] [PATCH for-2.8] block: Let write zeroes fallback work even with small max_transfer

On Tue, Nov 08, 2016 at 04:52:15PM -0600, Eric Blake wrote: > Commit 443668ca rewrote the write_zeroes logic to guarantee that > an unaligned request never crosses a cluster boundary. But > in the rewrite, the new code assumed that at most one iteration > would be needed to get to an alignment

Re: [Qemu-block] [PATCH] nbd: Don't inf-loop on early EOF

On 07/11/2016 23:45, Eric Blake wrote: > On 11/07/2016 04:22 PM, Max Reitz wrote: >> On 07.11.2016 21:38, Eric Blake wrote: >>> Commit 7d3123e converted a single read_sync() into a while loop >>> that assumed that read_sync() would either make progress or give >>> an error. But when the server

Re: [Qemu-block] [PATCH] Added iopmem device emulation

On Tue, Nov 08, 2016 at 09:46:47AM -0700, Logan Gunthorpe wrote: > On 08/11/16 08:58 AM, Stefan Hajnoczi wrote: > > My concern with the current implementation is that a PCI MMIO access > > invokes a synchronous blk_*() call. That can pause vcpu execution while > > I/O is happening and therefore

Re: [Qemu-block] [Qemu-devel] BdrvDirtyBitmap and bdrv_snapshot_goto

Am 08.11.2016 um 17:48 hat Vladimir Sementsov-Ogievskiy geschrieben: > 08.11.2016 15:18, Kevin Wolf wrote: > >Am 08.11.2016 um 12:08 hat Vladimir Sementsov-Ogievskiy geschrieben: > >>08.11.2016 14:05, Kevin Wolf wrote: > >>>Am 07.11.2016 um 17:10 hat Max Reitz geschrieben: > On 07.11.2016