Re: [Qemu-block] [PATCH 6/6] coroutine-lock: make CoRwlock thread-safe and fair

2017-02-15 Thread Fam Zheng
On Mon, 02/13 19:12, Paolo Bonzini wrote: > This adds a CoMutex around the existing CoQueue. Because the write-side s/CoQueue/CoRwlock/ > can just take CoMutex, the old "writer" field is not necessary anymore. > Instead of removing it altogether, count the number of pending writers > during a re

Re: [Qemu-block] [PATCH 0/6] Make CoMutex/CoQueue/CoRwlock thread-safe

2017-02-15 Thread Fam Zheng
On Mon, 02/13 19:12, Paolo Bonzini wrote: > This is yet another tiny bit of the multiqueue work, this time affecting > the synchronization infrastructure for coroutines. Currently, coroutines > synchronize between the main I/O thread and the dataplane iothread through > the AioContext lock. Howev

[Qemu-block] [PATCH v3 1/8] hw: Default -drive to if=ide explicitly where it works

2017-02-15 Thread Markus Armbruster
Block backends defined with -drive if=ide are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. if=ide drives not picked up that way can still be used with -device as if they had if=none, but that's unclean and best avoided. Unused

[Qemu-block] [PATCH v3 5/8] hw/arm/highbank: Default -drive to if=ide instead of if=scsi

2017-02-15 Thread Markus Armbruster
These machines have no onboard SCSI HBA, and no way to plug one. -drive if=scsi therefore cannot work. They do have an onboard IDE controller (sysbus-ahci), but fail to honor if=ide. Change their default to if=ide, and add a TODO comment on what needs to be done to actually honor -drive if=ide.

[Qemu-block] [PATCH v3 7/8] blockdev: Make orphaned -drive fatal

2017-02-15 Thread Markus Armbruster
Block backends defined with "-drive if=T" with T other than "none" are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. If machine initialization code doesn't comply, the block backend remains unused. This triggers a warning since

[Qemu-block] [PATCH v3 2/8] hw/arm/cubieboard hw/arm/xlnx-ep108: Fix units_per_default_bus

2017-02-15 Thread Markus Armbruster
Machine types cubieboard, xlnx-ep108, xlnx-zcu102 have an onboard AHCI controller, but neglect to set their MachineClass member units_per_default_bus = 1. This permits -drive if=ide,unit=1, which makes no sense for AHCI. It also screws up index=N for odd N, because it gets desugared to unit=1,bus

[Qemu-block] [PATCH v3 0/8] More sensible default for -drive interface type

2017-02-15 Thread Markus Armbruster
Block backends defined with -drive if=T, T!=none are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. if=T drives not picked up that way can still be used with -device as if they had if=none, but that's unclean and best avoided. Un

[Qemu-block] [PATCH v3 6/8] blockdev: Improve message for orphaned -drive

2017-02-15 Thread Markus Armbruster
We warn when a -drive isn't supported by the machine type (commit a66c9dc): $ qemu-system-x86_64 -S -display none -drive if=mtd Warning: Orphaned drive without device: id=mtd0,file=,if=mtd,bus=0,unit=0 Improve this to point to the offending bit of configuration: qemu-system-x86_64: -

[Qemu-block] [PATCH v3 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-15 Thread Markus Armbruster
We've traditionally rejected orphans here and there, but not systematically. For instance, the sun4m machines have an onboard SCSI HBA (bus=0), and have always rejected bus>0. Other machines with an onboard SCSI HBA don't. Commit a66c9dc made all orphans trigger a warning, and the previous commi

[Qemu-block] [PATCH v3 4/8] hw: Default -drive to if=none instead of scsi when scsi cannot work

2017-02-15 Thread Markus Armbruster
Block backends defined with -drive if=scsi are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. if=scsi drives not picked up that way can still be used with -device as if they had if=none, but that's unclean and best avoided. Unuse

[Qemu-block] [PATCH v3 3/8] hw: Default -drive to if=none instead of ide when ide cannot work

2017-02-15 Thread Markus Armbruster
Block backends defined with -drive if=ide are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. if=ide drives not picked up that way can still be used with -device as if they had if=none, but that's unclean and best avoided. Unused

[Qemu-block] [PATCH v15 04/25] tests: add hbitmap iter test

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Test that hbitmap iter is resistant to bitmap resetting. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/test-hbitmap.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/tests/test-hbitmap

[Qemu-block] [PATCH v15 16/25] qmp: add persistent flag to block-dirty-bitmap-add

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz --- blockdev.c | 18 +- qapi/block-core.json | 8 +++- 2 files changed, 24 ins

[Qemu-block] [PATCH v15 06/25] block/dirty-bitmap: add deserialize_ones func

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Add bdrv_dirty_bitmap_deserialize_ones() function, which is needed for qcow2 bitmap loading, to handle unallocated bitmap parts, marked as all-ones. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Kevin Wolf Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++ include

[Qemu-block] [PATCH v15 15/25] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_can_store_new_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 52 block/qcow2.c| 1 + block/qcow2.h| 4 3 files changed, 57 insertions(+) diff --git a/block/q

[Qemu-block] [PATCH v15 24/25] qmp: block-dirty-bitmap-remove: remove persistent

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Remove persistent bitmap from the storage on block-dirty-bitmap-remove. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 10 ++ 1 file changed, 10 insertions(+) diff --git a/blockdev.c b/blockdev.c index c41b791..a365cdf 100644

[Qemu-block] [PATCH v15 25/25] qcow2-bitmap: improve check_constraints_on_bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Add detailed error messages. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2-bitmap.c | 48 ++-- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/block/qcow2-bitmap.c b/block/qcow2-bitmap.c index 9177c56..e25c872 100644 --- a

[Qemu-block] [PATCH v15 01/25] specs/qcow2: fix bitmap granularity qemu-specific note

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow --- docs/specs/qcow2.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt index 80cdfd0..dda53dd 100644 --- a/docs/specs/qcow2.txt +++ b/docs/specs/qcow2.txt @@ -47

[Qemu-block] [PATCH v15 13/25] qcow2: add .bdrv_store_persistent_dirty_bitmaps()

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Realize block bitmap storing interface, to allow qcow2 images store persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 487 +-- block/qcow2.c| 1 + bl

[Qemu-block] [PATCH v15 00/25] qcow2: persistent dirty bitmaps

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Hi all! There is a new update of qcow2-bitmap series - v15. web: https://src.openvz.org/users/vsementsov/repos/qemu/browse?at=qcow2-bitmap-v15 git: https://src.openvz.org/scm/~vsementsov/qemu.git (tag qcow2-bitmap-v15) v15: 13,14: add John's r-b 15: qcow2_can_store_new_dirty_bitmap: - che

[Qemu-block] [PATCH v15 02/25] specs/qcow2: do not use wording 'bitmap header'

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
A bitmap directory entry is sometimes called a 'bitmap header'. This patch leaves only one name - 'bitmap directory entry'. The name 'bitmap header' creates misunderstandings with 'qcow2 header' and 'qcow2 bitmap header extension' (which is extension of qcow2 header) Signed-off-by: Vladimir Sement

[Qemu-block] [PATCH v15 07/25] qcow2: add bitmaps extension

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Add bitmap extension as specified in docs/specs/qcow2.txt. For now, just mirror extension header into Qcow2 state and check constraints. For now, disable image resize if it has bitmaps. It will be fixed later. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max R

[Qemu-block] [PATCH v15 09/25] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They are loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Extra data in bitmaps is not supported for now. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: Jo

[Qemu-block] [PATCH v15 12/25] block/dirty-bitmap: add bdrv_dirty_bitmap_next()

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 7 +++ include/block/dirty-bitmap.h | 3 +++ 2 files changed, 10 insertions(+) diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c index d2fbf55..9208844 100644

[Qemu-block] [PATCH v15 22/25] block/dirty-bitmap: add bdrv_remove_persistent_dirty_bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Interface for removing persistent bitmap from its storage. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 18 ++ include/block/block_int.h| 3 +++ include/block/dirty-bitmap.h | 3 +++ 3 files c

Re: [Qemu-block] [Qemu-devel] regarding bug name : -hda FAT:. limited to 504MBytes

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 23:24 hat Thomas Huth geschrieben: > On 14.02.2017 22:38, Shubham Kumar wrote: > > Since the problem seems like the used FAT-16 file system , > > Will it solve the problem if I change the code of vvfat.c for FAT-32 file > > system to increase acceptable file size ? > > As far a

[Qemu-block] [PATCH v15 18/25] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 5 + blockdev.c | 29 + include/block/dirty-bitmap.h | 2 ++ include/qemu/hbitmap.h | 8 qapi/block-co

[Qemu-block] [PATCH v15 19/25] iotests: test qcow2 persistent dirty bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- tests/qemu-iotests/165 | 89 ++ tests/qemu-iotests/165.out | 5 +++ tests/qemu-iotests/group | 1 + 3 files changed, 95 insertions(+) create mode 10

[Qemu-block] [PATCH v15 10/25] block/dirty-bitmap: add autoload field to BdrvDirtyBitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Mirror AUTO flag from Qcow2 bitmap in BdrvDirtyBitmap. This will be needed in future, to save this flag back to Qcow2 for persistent bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/dirty-bitmap.c | 15 +++ block/q

[Qemu-block] [PATCH v15 21/25] qcow2-bitmap: refcounts

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img check implementation. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 76 ++ block/qcow2-refcount.c |

[Qemu-block] [PATCH v15 23/25] qcow2: add .bdrv_remove_persistent_dirty_bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Realize .bdrv_remove_persistent_dirty_bitmap interface. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-bitmap.c | 41 + block/qcow2.c| 1 + block/qcow2.h| 3 +++ 3 files changed

[Qemu-block] [PATCH v15 05/25] block: fix bdrv_dirty_bitmap_granularity signature

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Make getter signature const-correct. This allows other functions with const dirty bitmap parameter use bdrv_dirty_bitmap_granularity(). Reviewed-by: Eric Blake Reviewed-by: John Snow Reviewed-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/dirty-bitmap.c | 2 +- i

[Qemu-block] [PATCH v15 03/25] hbitmap: improve dirty iter

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Make dirty iter resistant to resetting bits in corresponding HBitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- include/qemu/hbitmap.h | 26 -- util/hbitmap.c | 23 ++- 2 files changed, 26 i

[Qemu-block] [PATCH v15 08/25] block: introduce auto-loading bitmaps

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Auto loading bitmaps are bitmaps stored in the disk image, which should be loaded when the image is opened and become BdrvDirtyBitmaps for the corresponding drive. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz --- block.c | 14 +

[Qemu-block] [PATCH v15 11/25] block: introduce persistent dirty bitmaps

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
New field BdrvDirtyBitmap.persistent means, that bitmap should be saved on bdrv_close, using format driver. Format driver should maintain bitmap storing. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block.c | 32

[Qemu-block] [PATCH v15 14/25] block: add bdrv_can_store_new_dirty_bitmap

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
This will be needed to check some restrictions before making bitmap persistent in qmp-block-dirty-bitmap-add (this functionality will be added by future patch) Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block.c | 22 +

[Qemu-block] [PATCH v15 17/25] qmp: add autoload parameter to block-dirty-bitmap-add

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Optional. Default is false. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Denis V. Lunev Reviewed-by: Max Reitz Reviewed-by: John Snow --- blockdev.c | 18 -- qapi/block-core.json | 6 +- 2 files changed, 21 insertions(+), 3 deletions(-) diff --git

[Qemu-block] [PATCH v15 20/25] qcow2-refcount: rename inc_refcounts() and make it public

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
This is needed for the following patch, which will introduce refcounts checking for qcow2 bitmaps. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz Reviewed-by: John Snow --- block/qcow2-refcount.c | 53 ++ block/qcow2.h

[Qemu-block] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Currently backup to nbd target is broken, as nbd doesn't have .bdrv_get_info realization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: add WARNING === Since commit commit 4c9bca7e39a6e07ad02c1dcde3478363344ec60b Author: John Snow Date: Thu Feb 25 15:58:30 2016 -0500 block/backu

Re: [Qemu-block] [PATCH 6/6] coroutine-lock: make CoRwlock thread-safe and fair

2017-02-15 Thread Paolo Bonzini
On 15/02/2017 10:23, Fam Zheng wrote: > On Mon, 02/13 19:12, Paolo Bonzini wrote: >> This adds a CoMutex around the existing CoQueue. Because the write-side > > s/CoQueue/CoRwlock/ No, I meant that CoRwlock has a CoQueue, and after this patch it is wrapped by a CoMutex too. >> @@ -375,16 +38

Re: [Qemu-block] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread Fam Zheng
On Wed, 02/15 14:09, Vladimir Sementsov-Ogievskiy wrote: > Currently backup to nbd target is broken, as nbd doesn't have > .bdrv_get_info realization. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > v2: add WARNING > > === > > Since commit > > commit 4c9bca7e39a6e07ad02c1dcde3478363

Re: [Qemu-block] [Qemu-devel] [PATCH 6/6] coroutine-lock: make CoRwlock thread-safe and fair

2017-02-15 Thread Fam Zheng
On Wed, 02/15 12:20, Paolo Bonzini wrote: > > > On 15/02/2017 10:23, Fam Zheng wrote: > > On Mon, 02/13 19:12, Paolo Bonzini wrote: > >> This adds a CoMutex around the existing CoQueue. Because the write-side > > > > s/CoQueue/CoRwlock/ > > No, I meant that CoRwlock has a CoQueue, and after th

Re: [Qemu-block] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
15.02.2017 14:33, Fam Zheng wrote: On Wed, 02/15 14:09, Vladimir Sementsov-Ogievskiy wrote: Currently backup to nbd target is broken, as nbd doesn't have .bdrv_get_info realization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: add WARNING === Since commit commit 4c9bca7e39a6e07ad02c

Re: [Qemu-block] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread Fam Zheng
On Wed, 02/15 14:42, Vladimir Sementsov-Ogievskiy wrote: > 15.02.2017 14:33, Fam Zheng wrote: > > On Wed, 02/15 14:09, Vladimir Sementsov-Ogievskiy wrote: > > > Currently backup to nbd target is broken, as nbd doesn't have > > > .bdrv_get_info realization. > > > > > > Signed-off-by: Vladimir Semen

[Qemu-block] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi

2017-02-15 Thread Markus Armbruster
Drives defined with if=scsi get connected to buses created with -device, unlike other interface types. Deprecate this usage. There is no good default SCSI HBA for PC machines. Deprecate if=scsi there entirely. Before this series, frontends for -drive if=scsi get created by SCSI HBAs. Frontends

[Qemu-block] [PATCH v2 3/3] hw/i386: Deprecate -drive if=scsi with PC machine types

2017-02-15 Thread Markus Armbruster
The PC machines (pc-q35-* pc-i440fx-* pc-* isapc xenfv) automatically create lsi53c895a SCSI HBAs and SCSI devices to honor -drive if=scsi. For giggles, try -drive if=scsi,bus=25,media=cdrom --- this makes QEMU create 25 of them. lsi53c895a is thoroughly obsolete (PCI Ultra2 SCSI, ca. 2000), and c

[Qemu-block] [PATCH v2 1/3] hw/scsi: Concentrate -drive if=scsi auto-create in one place

2017-02-15 Thread Markus Armbruster
The logic to create frontends for -drive if=scsi is in SCSI HBAs. For all other interface types, it's in machine initialization code. A few machine types create the SCSI HBAs necessary for that. That's also not done for other interface types. I'm going to deprecate these SCSI eccentricities. I

[Qemu-block] [PATCH v2 2/3] hw: Deprecate -drive if=scsi with non-onboard HBAs

2017-02-15 Thread Markus Armbruster
Block backends defined with "-drive if=T" with T other than "none" are meant to be picked up by machine initialization code: a suitable frontend gets created and wired up automatically. Drives defined with if=scsi are also picked up by SCSI HBAs added with -device, unlike other interface types. D

Re: [Qemu-block] [PATCH v5 1/7] qcow2: Assert that cluster operations are aligned

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 20:25 hat Eric Blake geschrieben: > qcow2_discard_clusters() is set up to silently ignore sub-cluster > head or tail on unaligned requests. However, it is easy to audit > the various callers: qcow2_snapshot_create() has always passed > aligned data since the call was introduced in

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi

2017-02-15 Thread no-reply
Hi, Your series seems to have some coding style problems. See output below for more information: Subject: [Qemu-devel] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi Message-id: 1487161136-9018-1-git-send-email-arm...@redhat.com Type: series === TEST SCRIPT BEGIN === #!/bin/bash BASE=

Re: [Qemu-block] [Qemu-devel] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread no-reply
Hi, Your series failed automatic build test. Please find the testing commands and their output below. If you have docker installed, you can probably reproduce it locally. Subject: [Qemu-devel] [PATCH v2] backup: allow target without .bdrv_get_info Message-id: 1487156985-18321-1-git-send-email-vse

Re: [Qemu-block] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi

2017-02-15 Thread Paolo Bonzini
On 15/02/2017 13:18, Markus Armbruster wrote: > Drives defined with if=scsi get connected to buses created with > -device, unlike other interface types. Deprecate this usage. > > There is no good default SCSI HBA for PC machines. Deprecate if=scsi > there entirely. > > Before this series, fro

Re: [Qemu-block] [PATCH v2] backup: allow target without .bdrv_get_info

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
15.02.2017 14:09, Vladimir Sementsov-Ogievskiy wrote: Currently backup to nbd target is broken, as nbd doesn't have .bdrv_get_info realization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v2: add WARNING === Since commit commit 4c9bca7e39a6e07ad02c1dcde3478363344ec60b Author: John Snow

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi

2017-02-15 Thread Markus Armbruster
Paolo Bonzini writes: > On 15/02/2017 13:18, Markus Armbruster wrote: >> Drives defined with if=scsi get connected to buses created with >> -device, unlike other interface types. Deprecate this usage. >> >> There is no good default SCSI HBA for PC machines. Deprecate if=scsi >> there entirely.

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] hw: Deprecate unwanted use -drive if=scsi

2017-02-15 Thread Paolo Bonzini
On 15/02/2017 14:10, Markus Armbruster wrote: > Paolo Bonzini writes: > >> On 15/02/2017 13:18, Markus Armbruster wrote: >>> Drives defined with if=scsi get connected to buses created with >>> -device, unlike other interface types. Deprecate this usage. >>> >>> There is no good default SCSI HB

Re: [Qemu-block] [PATCH] block: Swap request limit definitions

2017-02-15 Thread Max Reitz
On 14.02.2017 10:52, Alberto Garcia wrote: > On Mon 13 Feb 2017 06:13:38 PM CET, Max Reitz wrote: > -#define BDRV_REQUEST_MAX_SECTORS MIN(SIZE_MAX >> BDRV_SECTOR_BITS, \ - INT_MAX >> BDRV_SECTOR_BITS) -#define BDRV_REQUEST_MAX_BYTES (BDRV_REQUEST_

Re: [Qemu-block] [PATCH v4 11/18] qcow2: convert QCow2 to use QCryptoBlock for encryption

2017-02-15 Thread Alberto Garcia
On Fri 10 Feb 2017 06:09:03 PM CET, Daniel P. Berrange wrote: > @@ -578,6 +582,7 @@ static void read_cache_sizes(BlockDriverState *bs, > QemuOpts *opts, > } > } > > + > typedef struct Qcow2ReopenState { > Qcow2Cache *l2_table_cache; > Qcow2Cache *refcount_block_cache; I don't k

Re: [Qemu-block] [RFC PATCH 01/41] block: Attach bs->file only during .bdrv_open()

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > The way that attaching bs->file worked was a bit unusual in that it was > the only child that would be attached to a node which is not opened yet. > Because of this, the block layer couldn't know yet which permissions the > driver would eventually need. > >

Re: [Qemu-block] [RFC PATCH 03/41] block: Add Error argument to bdrv_attach_child()

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > It will have to return an error soon, so prepare the callers for it. > > Signed-off-by: Kevin Wolf > --- > block.c | 13 ++--- > block/quorum.c| 8 +++- > include/block/block.h | 3 ++- > 3 files changed, 19 insertions(

Re: [Qemu-block] [Qemu-devel] [PATCH 05/18] nbd/client: fix drop_sync

2017-02-15 Thread Eric Blake
On 02/06/2017 05:17 PM, Eric Blake wrote: > On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> Comparison symbol is misused. It may lead to memory corruption. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- >> nbd/client.c | 2 +- >> 1 file changed, 1 insertion(+), 1 deletion

[Qemu-block] [PATCH v3] backup: allow target without .bdrv_get_info

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Currently backup to nbd target is broken, as nbd doesn't have .bdrv_get_info realization. Signed-off-by: Vladimir Sementsov-Ogievskiy --- v3: fix compilation (I feel like an idiot) adjust wording (Fam) v2: add WARNING === Since commit commit 4c9bca7e39a6e07ad02c1dcde3478363344ec60b A

Re: [Qemu-block] [PATCH v4 12/18] qcow2: extend specification to cover LUKS encryption

2017-02-15 Thread Alberto Garcia
On Fri 10 Feb 2017 06:09:04 PM CET, Daniel P. Berrange wrote: > Update the qcow2 specification to describe how the LUKS header is > placed inside a qcow2 file, when using LUKS encryption for the > qcow2 payload instead of the legacy AES-CBC encryption > > Reviewed-by: Max Reitz > Signed-off-by: Da

[Qemu-block] blockdev-add/del status

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
Hi all! What is the status of qmp commands blockdev-add and x-blockdev-del? Is blockdev-add stable (it never had x- prefix)? What should be done to get rid of "x-" for x-blockdev-del? I see a lot of work on blockdev-add in the list, but x-blockdev-del seems untouched about 8 moths.. -- Be

Re: [Qemu-block] [Qemu-devel] blockdev-add/del status

2017-02-15 Thread Eric Blake
On 02/15/2017 09:26 AM, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > What is the status of qmp commands blockdev-add and x-blockdev-del? Is > blockdev-add stable (it never had x- prefix)? What should be done to get > rid of "x-" for x-blockdev-del? We are still working on 'BlockdevOptions' f

Re: [Qemu-block] [PATCH v4 15/18] iotests: enable tests 134 and 158 to work with qcow (v1)

2017-02-15 Thread Alberto Garcia
On Fri 10 Feb 2017 06:09:07 PM CET, Daniel P. Berrange wrote: > The 138 and 158 iotests exercise the legacy qcow2 aes encryption > code path and they work fine with qcow v1 too. > > Reviewed-by: Max Reitz > Signed-off-by: Daniel P. Berrange Reviewed-by: Alberto Garcia Berto

Re: [Qemu-block] [PATCH v5 2/7] qcow2: Discard/zero clusters by byte count

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 20:25 hat Eric Blake geschrieben: > Passing a byte offset, but sector count, when we ultimately > want to operate on cluster granularity, is madness. Clean up > the external interfaces to take both offset and count as bytes, > while still keeping the assertion added previously tha

Re: [Qemu-block] [PATCH v5 4/7] blkdebug: Add pass-through write_zero and discard support

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 20:25 hat Eric Blake geschrieben: > In order to test the effects of artificial geometry constraints > on operations like write zero or discard, we first need blkdebug > to manage these actions. It also allows us to inject errors on > those operations, just like we can for read/wri

Re: [Qemu-block] [PATCH v5 5/7] blkdebug: Simplify override logic

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 20:25 hat Eric Blake geschrieben: > Rather than store into a local variable, then copy to the struct > if the value is valid, then reporting errors otherwise, it is > simpler to just store into the struct and report errors if the > value is invalid. This however requires that the

Re: [Qemu-block] blockdev-add/del status

2017-02-15 Thread Max Reitz
On 15.02.2017 16:26, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > What is the status of qmp commands blockdev-add and x-blockdev-del? Is > blockdev-add stable (it never had x- prefix)? What should be done to get > rid of "x-" for x-blockdev-del? To add to what Eric said, we simply forgot the

Re: [Qemu-block] [PATCH v5 6/7] blkdebug: Add ability to override unmap geometries

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 20:25 hat Eric Blake geschrieben: > Make it easier to simulate various unusual hardware setups (for > example, recent commits 3482b9b and b8d0a98 affect the Dell > Equallogic iSCSI with its 15M preferred and maximum unmap and > write zero sizing, or b2f95fe deals with the Linux loo

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] qemu-iotests: redirect nbd server stdout to /dev/null

2017-02-15 Thread Kevin Wolf
Am 14.02.2017 um 19:42 hat Eric Blake geschrieben: > On 02/14/2017 12:15 PM, Jeff Cody wrote: > > Some iotests (e.g. 174) try to filter the output of _make_test_image by > > piping the stdout. Pipe the server stdout to /dev/null, so that filter > > pipe does not need to wait until process completi

Re: [Qemu-block] [PATCH] block: Swap request limit definitions

2017-02-15 Thread Kevin Wolf
Am 15.02.2017 um 14:42 hat Max Reitz geschrieben: > On 14.02.2017 10:52, Alberto Garcia wrote: > > On Mon 13 Feb 2017 06:13:38 PM CET, Max Reitz wrote: > > > -#define BDRV_REQUEST_MAX_SECTORS MIN(SIZE_MAX >> BDRV_SECTOR_BITS, \ > - INT_MAX >> BDRV_SECT

Re: [Qemu-block] [PATCH] block: Swap request limit definitions

2017-02-15 Thread Max Reitz
On 15.02.2017 17:44, Kevin Wolf wrote: > Am 15.02.2017 um 14:42 hat Max Reitz geschrieben: >> On 14.02.2017 10:52, Alberto Garcia wrote: >>> On Mon 13 Feb 2017 06:13:38 PM CET, Max Reitz wrote: >>> >> -#define BDRV_REQUEST_MAX_SECTORS MIN(SIZE_MAX >> BDRV_SECTOR_BITS, \ >> -

Re: [Qemu-block] [PATCH 06/18] nbd/client: refactor drop_sync

2017-02-15 Thread Paolo Bonzini
On 08/02/2017 08:55, Vladimir Sementsov-Ogievskiy wrote: > 07.02.2017 02:19, Eric Blake wrote: >> On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: >>> Return 0 on success to simplify success checking. >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>> --- >>> nbd/client.c | 35 +

Re: [Qemu-block] [PATCH 07/18] nbd: Minimal structured read for client

2017-02-15 Thread Paolo Bonzini
On 07/02/2017 21:14, Eric Blake wrote: > On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> Minimal implementation: always send DF flag, to not deal with fragmented >> replies. > > This works well with your minimal server implementation, but I worry > that it will cause us to fall ov

Re: [Qemu-block] [PATCH 08/18] hbitmap: add next_zero function

2017-02-15 Thread Paolo Bonzini
On 07/02/2017 23:55, Eric Blake wrote: > On 02/03/2017 09:47 AM, Vladimir Sementsov-Ogievskiy wrote: >> The function searches for next zero bit. >> Also add interface for BdrvDirtyBitmap. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- >> block/dirty-bitmap.c | 5 + >> inc

Re: [Qemu-block] [PATCH 15/18] qmp: add block-dirty-bitmap-load

2017-02-15 Thread Paolo Bonzini
On 03/02/2017 16:47, Vladimir Sementsov-Ogievskiy wrote: > ## > +# @block-dirty-bitmap-load: > +# > +# Load a dirty bitmap from the storage (qcow2 file or nbd export) NBD export only in upstream QEMU? Paolo > +# Returns: nothing on success > +# If @node is not a valid block device, D

Re: [Qemu-block] [RFC PATCH 07/41] block: Default .bdrv_child_perm() for filter drivers

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > Most filters need permissions related to read and write for their > children, but only if the node has a parent that wants to use the same > operation on the filter. The same is true for resize. > > This adds a default implementation that simply forwards al

Re: [Qemu-block] [PATCH v3] backup: allow target without .bdrv_get_info

2017-02-15 Thread Jeff Cody
On Wed, Feb 15, 2017 at 05:58:13PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Currently backup to nbd target is broken, as nbd doesn't have > .bdrv_get_info realization. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > v3: fix compilation (I feel like an idiot) > adjust wording (F

Re: [Qemu-block] [PATCH 17/18] nbd: BLOCK_STATUS for standard get_block_status function: server part

2017-02-15 Thread Paolo Bonzini
On 09/02/2017 16:38, Eric Blake wrote: >> +static int blockstatus_to_extent(BlockDriverState *bs, uint64_t offset, >> + uint64_t length, NBDExtent *extent) >> +{ >> +BlockDriverState *file; >> +uint64_t start_sector = offset >> BDRV_SECTOR_BITS; >> +ui

Re: [Qemu-block] [PATCH v3] backup: allow target without .bdrv_get_info

2017-02-15 Thread Kevin Wolf
Am 15.02.2017 um 15:58 hat Vladimir Sementsov-Ogievskiy geschrieben: > Currently backup to nbd target is broken, as nbd doesn't have > .bdrv_get_info realization. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > > v3: fix compilation (I feel like an idiot) > adjust wording (Fam) >

Re: [Qemu-block] [PATCH 00/18] nbd: BLOCK_STATUS

2017-02-15 Thread Paolo Bonzini
On 03/02/2017 16:47, Vladimir Sementsov-Ogievskiy wrote: > Hi all! > > We really need exporting dirty bitmaps feature as well as remote > get_block_status for nbd devices. So, here is minimalistic and restricted > realization of 'structured reply' and 'block status' nbd protocol extension > (as

Re: [Qemu-block] [PATCH 17/18] nbd: BLOCK_STATUS for standard get_block_status function: server part

2017-02-15 Thread Paolo Bonzini
On 09/02/2017 16:38, Eric Blake wrote: > Umm, why are we sending only one status? If the client requests two ids > during NBD_OPT_SET_META_CONTEXT, we should be able to provide both > pieces of information at once. For a minimal implementation, it works > for proof of concept, but it is pretty r

Re: [Qemu-block] [PATCH 18/18] nbd: BLOCK_STATUS for standard get_block_status function: client part

2017-02-15 Thread Paolo Bonzini
On 09/02/2017 17:00, Eric Blake wrote: >> +if (!client->block_status_ok) { >> +*pnum = nb_sectors; >> +ret = BDRV_BLOCK_DATA | BDRV_BLOCK_ALLOCATED; >> +if (bs->drv->protocol_name) { This condition is always true, I think? >> +ret |= BDRV_BLOCK_OFFSET_VAL

Re: [Qemu-block] [RFC PATCH 09/41] block: Default .bdrv_child_perm() for format drivers

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > Almost all format drivers have the same characteristics as far as > permissions are concerned: They have one or more children for storing > their own data and, more importantly, metadata (can be written to and > grow even without external write requests, mus

Re: [Qemu-block] [PATCH] block: Swap request limit definitions

2017-02-15 Thread Max Reitz
On 15.02.2017 18:10, Kevin Wolf wrote: > Am 15.02.2017 um 17:48 hat Max Reitz geschrieben: >> On 15.02.2017 17:44, Kevin Wolf wrote: >>> Am 15.02.2017 um 14:42 hat Max Reitz geschrieben: On 14.02.2017 10:52, Alberto Garcia wrote: > On Mon 13 Feb 2017 06:13:38 PM CET, Max Reitz wrote: >

Re: [Qemu-block] [PATCH] block: Swap request limit definitions

2017-02-15 Thread Kevin Wolf
Am 15.02.2017 um 17:48 hat Max Reitz geschrieben: > On 15.02.2017 17:44, Kevin Wolf wrote: > > Am 15.02.2017 um 14:42 hat Max Reitz geschrieben: > >> On 14.02.2017 10:52, Alberto Garcia wrote: > >>> On Mon 13 Feb 2017 06:13:38 PM CET, Max Reitz wrote: > >>> > >> -#define BDRV_REQUEST_MAX_SECTOR

Re: [Qemu-block] [RFC PATCH 09/41] block: Default .bdrv_child_perm() for format drivers

2017-02-15 Thread Kevin Wolf
Am 15.02.2017 um 18:11 hat Max Reitz geschrieben: > On 13.02.2017 18:22, Kevin Wolf wrote: > > Almost all format drivers have the same characteristics as far as > > permissions are concerned: They have one or more children for storing > > their own data and, more importantly, metadata (can be writt

Re: [Qemu-block] [RFC PATCH 11/41] vvfat: Implement .bdrv_child_perm()

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > vvfat is the last remaining driver that can have children, but doesn't > implement .bdrv_child_perm() yet. The default handlers aren't suitable > here, so let's implement a very simple driver-specific one that protects > the internal child from being used by

Re: [Qemu-block] [RFC PATCH 09/41] block: Default .bdrv_child_perm() for format drivers

2017-02-15 Thread Max Reitz
On 15.02.2017 18:29, Kevin Wolf wrote: > Am 15.02.2017 um 18:11 hat Max Reitz geschrieben: >> On 13.02.2017 18:22, Kevin Wolf wrote: >>> Almost all format drivers have the same characteristics as far as >>> permissions are concerned: They have one or more children for storing >>> their own data and

Re: [Qemu-block] [RFC PATCH 10/41] block: Request child permissions in format drivers

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > This makes use of the .bdrv_child_perm() implementation for formats that > we just added. All format drivers expose the permissions they actually > need nows, so that they can be set accordingly and updated when parents > are attached or detached. > > The o

Re: [Qemu-block] [RFC PATCH 11/41] vvfat: Implement .bdrv_child_perm()

2017-02-15 Thread Kevin Wolf
Am 15.02.2017 um 18:30 hat Max Reitz geschrieben: > On 13.02.2017 18:22, Kevin Wolf wrote: > > vvfat is the last remaining driver that can have children, but doesn't > > implement .bdrv_child_perm() yet. The default handlers aren't suitable > > here, so let's implement a very simple driver-specific

Re: [Qemu-block] [RFC PATCH 13/41] block: Request real permissions in bdrv_attach_child()

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > Now that all block drivers with children tell us what permissions they > need from each of their children, bdrv_attach_child() can use this > information and make the right requirements while trying to attach new > children. > > Signed-off-by: Kevin Wolf >

Re: [Qemu-block] [RFC PATCH 14/41] block: Add permissions to BlockBackend

2017-02-15 Thread Max Reitz
On 13.02.2017 18:22, Kevin Wolf wrote: > The BlockBackend can now store the permissions that its user requires. > This is necessary because nodes can be ejected from or inserted into a > BlockBackend and all of these operations must make sure that the user > still gets what it requested initially.

Re: [Qemu-block] [PATCH v3 8/8] hw: Drop superfluous special checks for orphaned -drive

2017-02-15 Thread John Snow
On 02/15/2017 05:05 AM, Markus Armbruster wrote: > We've traditionally rejected orphans here and there, but not > systematically. For instance, the sun4m machines have an onboard SCSI > HBA (bus=0), and have always rejected bus>0. Other machines with an > onboard SCSI HBA don't. > > Commit a66

Re: [Qemu-block] [Qemu-devel] [PATCH v15 15/25] qcow2: add .bdrv_can_store_new_dirty_bitmap

2017-02-15 Thread John Snow
On 02/15/2017 05:10 AM, Vladimir Sementsov-Ogievskiy wrote: > Realize .bdrv_can_store_new_dirty_bitmap interface. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Thanks, Reviewed-by: John Snow > --- > block/qcow2-bitmap.c | 52 > > bloc

Re: [Qemu-block] [Qemu-devel] [PATCH v15 16/25] qmp: add persistent flag to block-dirty-bitmap-add

2017-02-15 Thread John Snow
On 02/15/2017 05:10 AM, Vladimir Sementsov-Ogievskiy wrote: > Add optional 'persistent' flag to qmp command block-dirty-bitmap-add. > Default is false. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Signed-off-by: Denis V. Lunev > Reviewed-by: Max Reitz Reviewed-by: John Snow > --- > b

Re: [Qemu-block] [Qemu-devel] [PATCH v15 25/25] qcow2-bitmap: improve check_constraints_on_bitmap

2017-02-15 Thread John Snow
On 02/15/2017 05:10 AM, Vladimir Sementsov-Ogievskiy wrote: > Add detailed error messages. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow > --- > block/qcow2-bitmap.c | 48 ++-- > 1 file changed, 34 insertions(+), 14 deleti

Re: [Qemu-block] [Qemu-devel] [PATCH 14/17] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-15 Thread John Snow
On 02/13/2017 04:54 AM, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Max Reitz This is simply the same as the version in the other two series, right? Reviewed-by: John Snow > --- > block/dirty-bitmap.c | 5 + > blockdev.c

Re: [Qemu-block] [Qemu-devel] [PATCH 14/17] qmp: add x-debug-block-dirty-bitmap-sha256

2017-02-15 Thread Vladimir Sementsov-Ogievskiy
16.02.2017 03:35, John Snow wrote: On 02/13/2017 04:54 AM, Vladimir Sementsov-Ogievskiy wrote: Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz This is simply the same as the version in the other two series, right? Yes. Context a bit differs... Aha, I've discovered that i

  1   2   >