[Qemu-block] [PATCH v3 21/23] block: Align block status requests

2017-07-13 Thread Eric Blake
Any device that has request_alignment greater than 512 should be unable to report status at a finer granularity; it may also be simpler for such devices to be guaranteed that the block layer has rounded things out to the granularity boundary (the way the block layer already rounds all other I/O

[Qemu-block] [PATCH v3 17/23] qemu-img: Change check_empty_sectors() to byte-based

2017-07-13 Thread Eric Blake
Continue on the quest to make more things byte-based instead of sector-based. Signed-off-by: Eric Blake --- v3: new patch --- qemu-img.c | 27 +++ 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/qemu-img.c b/qemu-img.c index

[Qemu-block] [PATCH v3 10/23] block: Switch bdrv_common_block_status_above() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change). Signed-off-by: Eric Blake Reviewed-by: Fam Zheng --- v3: rebase to allocation/mapping sense

[Qemu-block] [PATCH v3 16/23] qemu-img: Drop redundant error message in compare

2017-07-13 Thread Eric Blake
If a read error is encountered during 'qemu-img compare', we were printing the "Error while reading offset ..." message twice. Update the testsuite for the improved output. Further simplify the code by hoisting the error code conversion into the helper function, rather than repeating it at the

[Qemu-block] [PATCH v2 10/29] vvfat: use DIV_ROUND_UP

2017-07-13 Thread Marc-André Lureau
I used the clang-tidy qemu-round check to generate the fix: https://github.com/elmarco/clang-tools-extra Signed-off-by: Marc-André Lureau --- block/vvfat.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index

[Qemu-block] [PATCH v2 08/29] qcow2: use DIV_ROUND_UP

2017-07-13 Thread Marc-André Lureau
I used the clang-tidy qemu-round check to generate the fix: https://github.com/elmarco/clang-tools-extra Signed-off-by: Marc-André Lureau --- block/qcow2-cluster.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/qcow2-cluster.c

[Qemu-block] [PATCH v3 11/23] block: Switch bdrv_co_get_block_status_above() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal type (no semantic change), and rename it to match the corresponding public function rename. Signed-off-by: Eric Blake Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v3 18/23] qemu-img: Change compare_sectors() to be byte-based

2017-07-13 Thread Eric Blake
In the continuing quest to make more things byte-based, change compare_sectors(), renaming it to compare_buffers() in the process. Note that one caller (qemu-img compare) only cares about the first difference, while the other (qemu-img rebase) cares about how many consecutive sectors have the

[Qemu-block] [PATCH v2 09/29] vpc: use DIV_ROUND_UP

2017-07-13 Thread Marc-André Lureau
I used the clang-tidy qemu-round check to generate the fix: https://github.com/elmarco/clang-tools-extra Signed-off-by: Marc-André Lureau --- block/vpc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/vpc.c b/block/vpc.c index

Re: [Qemu-block] [PATCH v5 3/9] nbd: Expose and debug more NBD constants

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
07.07.2017 23:30, Eric Blake wrote: The NBD protocol has several constants defined in various extensions that we are about to implement. Expose them to the code, along with an easy way to map various constants to strings during diagnostic messages. Signed-off-by: Eric Blake

Re: [Qemu-block] [Qemu-devel] [PULL 00/85] Block layer patches

2017-07-13 Thread Peter Maydell
On 11 July 2017 at 17:06, Max Reitz wrote: > The following changes since commit 29741be341d50e4311e42ca3199f9b0bcfd4f5d2: > > Merge remote-tracking branch > 'remotes/awilliam/tags/vfio-updates-20170710.0' into staging (2017-07-11 > 13:47:28 +0100) > > are available in the

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 03:49:09PM +0200, Peter Lieven wrote: > Am 13.07.2017 um 11:21 schrieb Daniel P. Berrange: > > On Thu, Jul 13, 2017 at 10:44:53AM +0200, Peter Lieven wrote: > > > Am 10.07.2017 um 15:55 schrieb Daniel P. Berrange: > > > > On Mon, Jul 10, 2017 at 03:52:18PM +0200, Kevin Wolf

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 04:03:47PM +0200, Peter Lieven wrote: > Am 13.07.2017 um 16:00 schrieb Daniel P. Berrange: > > On Thu, Jul 13, 2017 at 03:49:09PM +0200, Peter Lieven wrote: > > > Am 13.07.2017 um 11:21 schrieb Daniel P. Berrange: > > > > On Thu, Jul 13, 2017 at 10:44:53AM +0200, Peter

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 05:13:23PM +0200, Peter Lieven wrote: > Am 13.07.2017 um 17:06 schrieb Daniel P. Berrange: > > On Thu, Jul 13, 2017 at 05:02:51PM +0200, Peter Lieven wrote: > > > Am 13.07.2017 um 17:01 schrieb Daniel P. Berrange: > > > > On Thu, Jul 13, 2017 at 05:00:39PM +0200, Peter

[Qemu-block] [PATCH v6 1/4] block: pass bdrv_* methods to bs->file by default in block filters

2017-07-13 Thread Manos Pitsidianakis
The following functions fail if bs->drv is a filter and does not implement them: bdrv_probe_blocksizes bdrv_probe_geometry bdrv_truncate bdrv_has_zero_init bdrv_get_info Instead, the call should be passed to bs->file if it exists, to allow filter drivers to support those methods without

[Qemu-block] [PATCH v3 09/23] block: Switch BdrvCoGetBlockStatusData to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal type (no semantic change), and rename it to match the corresponding public function rename. Signed-off-by: Eric Blake Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v3 12/23] block: Convert bdrv_get_block_status_above() to bytes

2017-07-13 Thread Eric Blake
We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned

[Qemu-block] [PATCH v3 22/23] block: Relax bdrv_aligned_preadv() assertion

2017-07-13 Thread Eric Blake
Now that bdrv_is_allocated accepts non-aligned inputs, we can remove the TODO added in commit d6a644bb. Signed-off-by: Eric Blake --- v3: new patch [Kevin] --- block/io.c | 8 ++-- 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/block/io.c b/block/io.c

Re: [Qemu-block] [PATCH v5 3/4] qcow2: add shrink image support

2017-07-13 Thread Pavel Butsykin
On 13.07.2017 17:36, Max Reitz wrote: On 2017-07-13 10:41, Kevin Wolf wrote: Am 12.07.2017 um 18:58 hat Max Reitz geschrieben: On 2017-07-12 16:52, Kevin Wolf wrote: Am 12.07.2017 um 13:46 hat Pavel Butsykin geschrieben: This patch add shrinking of the image file for qcow2. As a result, this

[Qemu-block] [PATCH v3 14/23] qemu-img: Speed up compare on pre-allocated larger file

2017-07-13 Thread Eric Blake
Compare the following images with all-zero contents: $ truncate --size 1M A $ qemu-img create -f qcow2 -o preallocation=off B 1G $ qemu-img create -f qcow2 -o preallocation=metadata C 1G On my machine, the difference is noticeable for pre-patch speeds, with more than an order of magnitude in

Re: [Qemu-block] [PATCH v5 3/4] qcow2: add shrink image support

2017-07-13 Thread Pavel Butsykin
On 13.07.2017 11:41, Kevin Wolf wrote: Am 12.07.2017 um 18:58 hat Max Reitz geschrieben: On 2017-07-12 16:52, Kevin Wolf wrote: Am 12.07.2017 um 13:46 hat Pavel Butsykin geschrieben: This patch add shrinking of the image file for qcow2. As a result, this allows us to reduce the virtual image

Re: [Qemu-block] [PATCH v5 2/4] qcow2: add qcow2_cache_discard

2017-07-13 Thread Pavel Butsykin
On 12.07.2017 17:45, Kevin Wolf wrote: Am 12.07.2017 um 13:46 hat Pavel Butsykin geschrieben: Whenever l2/refcount table clusters are discarded from the file we can automatically drop unnecessary content of the cache tables. This reduces the chance of eviction useful cache data and eliminates

Re: [Qemu-block] [PATCH 29/31] vpc: Switch to .bdrv_co_block_status()

2017-07-13 Thread Eric Blake
On 07/13/2017 07:55 AM, Kevin Wolf wrote: > Am 18.04.2017 um 03:33 hat Eric Blake geschrieben: >> We are gradually moving away from sector-based interfaces, towards >> byte-based. Update the vpc driver accordingly. >> >> Signed-off-by: Eric Blake > > I know this is an old

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 16:07 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 04:03:47PM +0200, Peter Lieven wrote: Am 13.07.2017 um 16:00 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 03:49:09PM +0200, Peter Lieven wrote: Am 13.07.2017 um 11:21 schrieb Daniel P. Berrange: On Thu, Jul 13,

Re: [Qemu-block] [Qemu-devel] [PATCH v5 5/9] nbd: Refactor reply to NBD_OPT_EXPORT_NAME

2017-07-13 Thread Eric Blake
On 07/07/2017 03:30 PM, Eric Blake wrote: > Reply directly in nbd_negotiate_handle_export_name(), rather than > waiting until nbd_negotiate_options() completes. This will make it > easier to implement NBD_OPT_GO. Pass additional parameters around, > rather than stashing things inside NBDClient.

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 17:17 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 05:13:23PM +0200, Peter Lieven wrote: Am 13.07.2017 um 17:06 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 05:02:51PM +0200, Peter Lieven wrote: Am 13.07.2017 um 17:01 schrieb Daniel P. Berrange: On Thu, Jul 13,

[Qemu-block] [PATCH v6 0/4] block: Block driver callbacks fixes

2017-07-13 Thread Manos Pitsidianakis
This series makes implementing some of the bdrv_* callbacks easier for block filters by passing requests to bs->file if bs->drv doesn't implement it instead of failing, and adding default bdrv_co_get_block_status() implementations. This is based against commit ced1484322 of

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 17:06 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 05:02:51PM +0200, Peter Lieven wrote: Am 13.07.2017 um 17:01 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 05:00:39PM +0200, Peter Lieven wrote: Am 13.07.2017 um 16:58 schrieb Daniel P. Berrange: On Thu, Jul 13,

[Qemu-block] [PATCH v3 02/23] block: Add flag to avoid wasted work in bdrv_is_allocated()

2017-07-13 Thread Eric Blake
Not all callers care about which BDS owns the mapping for a given range of the file. In particular, bdrv_is_allocated() cares more about finding the largest run of allocated data from the guest perspective, whether or not that data is consecutive from the host perspective. Therefore, doing

[Qemu-block] [PATCH v3 06/23] qemu-img: Switch get_block_status() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Continue by converting an internal function (no semantic change), and simplifying its caller accordingly. Signed-off-by: Eric Blake Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v3 00/23] make bdrv_get_block_status byte-based

2017-07-13 Thread Eric Blake
There are patches floating around to add NBD_CMD_BLOCK_STATUS, but NBD wants to report status on byte granularity (even if the reporting will probably be naturally aligned to sectors or even much higher levels). I've therefore started the task of converting our block status code to report at a

[Qemu-block] [PATCH v3 01/23] block: Allow NULL file for bdrv_get_block_status()

2017-07-13 Thread Eric Blake
Not all callers care about which BDS owns the mapping for a given range of the file. This patch merely simplifies the callers by consolidating the logic in the common call point, while guaranteeing a non-NULL file to all the driver callbacks, for no semantic change. The only caller that does not

[Qemu-block] [PATCH v3 03/23] block: Make bdrv_round_to_clusters() signature more useful

2017-07-13 Thread Eric Blake
In the process of converting sector-based interfaces to bytes, I'm finding it easier to represent a byte count as a 64-bit integer at the block layer (even if we are internally capped by SIZE_MAX or even INT_MAX for individual transactions, it's still nicer to not have to worry about

[Qemu-block] [PATCH v3 04/23] qcow2: Switch is_zero_sectors() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change), and rename it to is_zero() in the process. Signed-off-by: Eric Blake Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v3 05/23] block: Switch bdrv_make_zero() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Change the internal loop iteration of zeroing a device to track by bytes instead of sectors (although we are still guaranteed that we iterate by steps that are sector-aligned).

Re: [Qemu-block] [PATCH v5 1/9] nbd: Create struct for tracking export info

2017-07-13 Thread Eric Blake
On 07/13/2017 07:31 AM, Vladimir Sementsov-Ogievskiy wrote: > 13.07.2017 15:09, Vladimir Sementsov-Ogievskiy wrote: >> 07.07.2017 23:30, Eric Blake wrote: >>> The NBD Protocol is introducing some additional information >>> about exports, such as minimum request size and alignment, as >>> well as

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 17:01 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 05:00:39PM +0200, Peter Lieven wrote: Am 13.07.2017 um 16:58 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 04:18:13PM +0200, Peter Lieven wrote: Okay, so it has to be a mix of QAPI parsing and manual parameter

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 05:00:39PM +0200, Peter Lieven wrote: > Am 13.07.2017 um 16:58 schrieb Daniel P. Berrange: > > On Thu, Jul 13, 2017 at 04:18:13PM +0200, Peter Lieven wrote: > > > Okay, so it has to be a mix of QAPI parsing and manual parameter checking, > > > right? > > Yeah. It does feel

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Eric Blake
On 07/13/2017 10:13 AM, Peter Lieven wrote: >> >>> I currently have the following: >>> >>>options = qemu_opts_to_qdict(opts, NULL); >>>qdict_extract_subqdict(options, , "compress."); Can you show your .json patches as well? > Yes, but

[Qemu-block] [PATCH v6 4/4] block: add default implementations for bdrv_co_get_block_status()

2017-07-13 Thread Manos Pitsidianakis
bdrv_co_get_block_status_from_file() and bdrv_co_get_block_status_from_backing() set *file to bs->file and bs->backing respectively, so that bdrv_co_get_block_status() can recurse to them. Future block drivers won't have to duplicate code to implement this. Reviewed-by: Fam Zheng

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 11:21 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 10:44:53AM +0200, Peter Lieven wrote: Am 10.07.2017 um 15:55 schrieb Daniel P. Berrange: On Mon, Jul 10, 2017 at 03:52:18PM +0200, Kevin Wolf wrote: Am 10.07.2017 um 15:44 hat Daniel P. Berrange geschrieben: On Mon,

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 16:00 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 03:49:09PM +0200, Peter Lieven wrote: Am 13.07.2017 um 11:21 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 10:44:53AM +0200, Peter Lieven wrote: Am 10.07.2017 um 15:55 schrieb Daniel P. Berrange: On Mon, Jul 10,

Re: [Qemu-block] [PATCH v5 3/4] qcow2: add shrink image support

2017-07-13 Thread Max Reitz
On 2017-07-13 10:41, Kevin Wolf wrote: > Am 12.07.2017 um 18:58 hat Max Reitz geschrieben: >> On 2017-07-12 16:52, Kevin Wolf wrote: >>> Am 12.07.2017 um 13:46 hat Pavel Butsykin geschrieben: This patch add shrinking of the image file for qcow2. As a result, this allows us to

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 13.07.2017 um 16:58 schrieb Daniel P. Berrange: On Thu, Jul 13, 2017 at 04:18:13PM +0200, Peter Lieven wrote: Okay, so it has to be a mix of QAPI parsing and manual parameter checking, right? Yeah. It does feel like a valid RFE for QAPI to add a permitted range to 'int' types though, which

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 04:18:13PM +0200, Peter Lieven wrote: > > Okay, so it has to be a mix of QAPI parsing and manual parameter checking, > right? Yeah. It does feel like a valid RFE for QAPI to add a permitted range to 'int' types though, which would simplify the code in future. > I

[Qemu-block] [PULL 2/9] block: Don't try to set *errp directly

2017-07-13 Thread Markus Armbruster
From: Eduardo Habkost Assigning directly to *errp is not valid, as errp may be NULL, _fatal, or _abort. Use error_propagate() instead. With this, there's no need to check if errp is NULL anymore, as error_propagate() and error_prepend() are able to handle that. Cc: Kevin

Re: [Qemu-block] [PATCH v5 1/9] nbd: Create struct for tracking export info

2017-07-13 Thread Paolo Bonzini
On 13/07/2017 16:35, Eric Blake wrote: > On 07/13/2017 07:31 AM, Vladimir Sementsov-Ogievskiy wrote: >> 13.07.2017 15:09, Vladimir Sementsov-Ogievskiy wrote: >>> 07.07.2017 23:30, Eric Blake wrote: The NBD Protocol is introducing some additional information about exports, such as minimum

[Qemu-block] [PATCH v6 2/4] block: remove unused bdrv_media_changed

2017-07-13 Thread Manos Pitsidianakis
This function is not used anywhere, so remove it. Markus Armbruster adds: The i82078 floppy device model used to call bdrv_media_changed() to implement its media change bit when backed by a host floppy. This went away in 21fcf36 "fdc: simplify media change handling". Probably broke host floppy

[Qemu-block] [PATCH v6 3/4] block: remove bdrv_truncate callback in blkdebug

2017-07-13 Thread Manos Pitsidianakis
Now that bdrv_truncate is passed to bs->file by default, remove the callback from block/blkdebug.c and set is_filter to true. is_filter also gives access to other callbacks that are forwarded automatically to bs->file for filters. Reviewed-by: Eric Blake Reviewed-by: Stefan

Re: [Qemu-block] [Qemu-devel] [PATCH v5 14/18] qcow2: Switch qcow2_measure() to byte-based iteration

2017-07-13 Thread John Snow
On 07/12/2017 09:05 PM, Eric Blake wrote: > This is new code, but it is easier to read if it makes passes over > the image using bytes rather than sectors (and will get easier in > the future when bdrv_get_block_status is converted to byte-based). > > While at it, fix a bug in the computation

Re: [Qemu-block] [Qemu-devel] [PATCH v5 13/18] mirror: Switch mirror_dirty_init() to byte-based iteration

2017-07-13 Thread John Snow
On 07/12/2017 09:05 PM, Eric Blake wrote: > Now that we have adjusted the majority of the calls this function > makes to be byte-based, it is easier to read the code if it makes > passes over the image using bytes rather than sectors. > > Signed-off-by: Eric Blake > Looks

Re: [Qemu-block] [Qemu-devel] [PATCH v2 10/29] vvfat: use DIV_ROUND_UP

2017-07-13 Thread Eric Blake
On 07/13/2017 11:32 AM, Marc-André Lureau wrote: > I used the clang-tidy qemu-round check to generate the fix: > https://github.com/elmarco/clang-tools-extra > > Signed-off-by: Marc-André Lureau > --- > block/vvfat.c | 4 ++-- > 1 file changed, 2 insertions(+), 2

Re: [Qemu-block] [Qemu-devel] [PATCH v2 08/29] qcow2: use DIV_ROUND_UP

2017-07-13 Thread Eric Blake
On 07/13/2017 11:31 AM, Marc-André Lureau wrote: > I used the clang-tidy qemu-round check to generate the fix: > https://github.com/elmarco/clang-tools-extra > > Signed-off-by: Marc-André Lureau > --- > block/qcow2-cluster.c | 2 +- > 1 file changed, 1 insertion(+),

Re: [Qemu-block] [Qemu-devel] [PATCH v2 09/29] vpc: use DIV_ROUND_UP

2017-07-13 Thread Eric Blake
On 07/13/2017 11:31 AM, Marc-André Lureau wrote: > I used the clang-tidy qemu-round check to generate the fix: > https://github.com/elmarco/clang-tools-extra > > Signed-off-by: Marc-André Lureau > --- > block/vpc.c | 4 ++-- > 1 file changed, 2 insertions(+), 2

Re: [Qemu-block] [Qemu-devel] [PATCH v2 03/29] vhdx: use QEMU_ALIGN_DOWN

2017-07-13 Thread Eric Blake
On 07/13/2017 11:31 AM, Marc-André Lureau wrote: > I used the clang-tidy qemu-round check to generate the fix: > https://github.com/elmarco/clang-tools-extra > > Signed-off-by: Marc-André Lureau > --- > block/vhdx-log.c | 2 +- > 1 file changed, 1 insertion(+), 1

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/9] block: -device drive= fixes

2017-07-13 Thread John Snow
On 07/13/2017 07:41 AM, Kevin Wolf wrote: > Since 2.9 it is possible for a user to care only about nodes and qdev devices, > but not about BlockBackends, by defining with -blockdev and using their node > name in -device. Devices create an anonymous BlockBackend internally then. > > One of the

[Qemu-block] [PATCH v3 08/23] block: Switch bdrv_co_get_block_status() to byte-based

2017-07-13 Thread Eric Blake
We are gradually converting to byte-based interfaces, as they are easier to reason about than sector-based. Convert another internal function (no semantic change); and as with its public counterpart, rename to bdrv_co_block_status() to make the compiler enforce that we catch all uses. For now,

[Qemu-block] [PATCH v3 07/23] block: Convert bdrv_get_block_status() to bytes

2017-07-13 Thread Eric Blake
We are gradually moving away from sector-based interfaces, towards byte-based. In the common case, allocation is unlikely to ever use values that are not naturally sector-aligned, but it is possible that byte-based values will let us be more precise about allocation at the end of an unaligned

[Qemu-block] [PATCH v3 13/23] qemu-img: Simplify logic in img_compare()

2017-07-13 Thread Eric Blake
As long as we are querying the status for a chunk smaller than the known image size, we are guaranteed that a successful return will have set pnum to a non-zero size (pnum is zero only for queries beyond the end of the file). Use that to slightly simplify the calculation of the current chunk size

[Qemu-block] [PATCH v3 15/23] qemu-img: Add find_nonzero()

2017-07-13 Thread Eric Blake
During 'qemu-img compare', when we are checking that an allocated portion of one file is all zeros, we don't need to waste time computing how many additional sectors after the first non-zero byte are also non-zero. Create a new helper find_nonzero() to do the check for a first non-zero sector,

[Qemu-block] [PATCH v3 19/23] qemu-img: Change img_rebase() to be byte-based

2017-07-13 Thread Eric Blake
In the continuing quest to make more things byte-based, change the internal iteration of img_rebase(). We can finally drop the TODO assertion added earlier, now that the entire algorithm is byte-based and no longer has to shift from bytes to sectors. Most of the change is mechanical

[Qemu-block] [PATCH v3 23/23] qemu-io: Relax 'alloc' now that block-status doesn't assert

2017-07-13 Thread Eric Blake
Previously, the alloc command required that input parameters be sector-aligned and clamped to 32 bits, because the underlying bdrv_is_allocated used a 32-bit parameter and asserted aligned inputs. But now that we have fixed block status to report a 64-bit bytes value, and to properly round

[Qemu-block] [PATCH v3 20/23] qemu-img: Change img_compare() to be byte-based

2017-07-13 Thread Eric Blake
In the continuing quest to make more things byte-based, change the internal iteration of img_compare(). We can finally drop the TODO assertion added earlier, now that the entire algorithm is byte-based and no longer has to shift from bytes to sectors. Most of the change is mechanical

[Qemu-block] [PATCH v2 07/29] dmg: use DIV_ROUND_UP

2017-07-13 Thread Marc-André Lureau
I used the clang-tidy qemu-round check to generate the fix: https://github.com/elmarco/clang-tools-extra Signed-off-by: Marc-André Lureau Reviewed-by: Stefan Hajnoczi --- block/dmg.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[Qemu-block] [PATCH v2 03/29] vhdx: use QEMU_ALIGN_DOWN

2017-07-13 Thread Marc-André Lureau
I used the clang-tidy qemu-round check to generate the fix: https://github.com/elmarco/clang-tools-extra Signed-off-by: Marc-André Lureau --- block/vhdx-log.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/vhdx-log.c b/block/vhdx-log.c

Re: [Qemu-block] [Qemu-devel] [PATCH v5 05/18] dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes

2017-07-13 Thread John Snow
On 07/12/2017 09:05 PM, Eric Blake wrote: > We are still using an internal hbitmap that tracks a size in sectors, > with the granularity scaled down accordingly, because it lets us > use a shortcut for our iterators which are currently sector-based. > But there's no reason we can't track the

Re: [Qemu-block] [PATCH V2 2/8] qapi: add compress parameters to Qcow2 Blockdev options

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 10:52:41AM +0200, Kevin Wolf wrote: > Am 13.07.2017 um 10:45 hat Peter Lieven geschrieben: > > Am 10.07.2017 um 15:30 schrieb Kevin Wolf: > > >Am 10.07.2017 um 15:24 hat Peter Lieven geschrieben: > > >>Am 10.07.2017 um 15:10 schrieb Kevin Wolf: > > >>>Am 29.06.2017 um 12:57

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Peter Lieven
Am 10.07.2017 um 15:55 schrieb Daniel P. Berrange: On Mon, Jul 10, 2017 at 03:52:18PM +0200, Kevin Wolf wrote: Am 10.07.2017 um 15:44 hat Daniel P. Berrange geschrieben: On Mon, Jul 10, 2017 at 03:34:59PM +0200, Kevin Wolf wrote: Am 10.07.2017 um 15:29 hat Peter Lieven geschrieben: Am

Re: [Qemu-block] [PATCH V2 2/8] qapi: add compress parameters to Qcow2 Blockdev options

2017-07-13 Thread Peter Lieven
Am 10.07.2017 um 15:30 schrieb Kevin Wolf: Am 10.07.2017 um 15:24 hat Peter Lieven geschrieben: Am 10.07.2017 um 15:10 schrieb Kevin Wolf: Am 29.06.2017 um 12:57 hat Peter Lieven geschrieben: Signed-off-by: Peter Lieven --- qapi/block-core.json | 44

Re: [Qemu-block] [Qemu-devel] [PATCH v4 14/17] qcow2: Switch load_bitmap_data() to byte-based iteration

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
03.07.2017 18:10, Eric Blake wrote: Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by: Eric Blake --- v4: new patch ---

Re: [Qemu-block] [PATCH V2 5/8] block/qcow2: read and write the compress format extension

2017-07-13 Thread Daniel P. Berrange
On Thu, Jul 13, 2017 at 10:44:53AM +0200, Peter Lieven wrote: > Am 10.07.2017 um 15:55 schrieb Daniel P. Berrange: > > On Mon, Jul 10, 2017 at 03:52:18PM +0200, Kevin Wolf wrote: > > > Am 10.07.2017 um 15:44 hat Daniel P. Berrange geschrieben: > > > > On Mon, Jul 10, 2017 at 03:34:59PM +0200,

Re: [Qemu-block] [PATCH v5 3/4] qcow2: add shrink image support

2017-07-13 Thread Kevin Wolf
Am 12.07.2017 um 18:58 hat Max Reitz geschrieben: > On 2017-07-12 16:52, Kevin Wolf wrote: > > Am 12.07.2017 um 13:46 hat Pavel Butsykin geschrieben: > >> This patch add shrinking of the image file for qcow2. As a result, this > >> allows > >> us to reduce the virtual image size and free up space

Re: [Qemu-block] [PATCH V2 2/8] qapi: add compress parameters to Qcow2 Blockdev options

2017-07-13 Thread Kevin Wolf
Am 13.07.2017 um 10:45 hat Peter Lieven geschrieben: > Am 10.07.2017 um 15:30 schrieb Kevin Wolf: > >Am 10.07.2017 um 15:24 hat Peter Lieven geschrieben: > >>Am 10.07.2017 um 15:10 schrieb Kevin Wolf: > >>>Am 29.06.2017 um 12:57 hat Peter Lieven geschrieben: > Signed-off-by: Peter Lieven

[Qemu-block] [PATCH v5 4/4] block: add default implementations for bdrv_co_get_block_status()

2017-07-13 Thread Manos Pitsidianakis
bdrv_co_get_block_status_from_file() and bdrv_co_get_block_status_from_backing() set *file to bs->file and bs->backing respectively, so that bdrv_co_get_block_status() can recurse to them. Future block drivers won't have to duplicate code to implement this. Reviewed-by: Fam Zheng

[Qemu-block] [PATCH v5 0/4] block: Block driver callbacks fixes

2017-07-13 Thread Manos Pitsidianakis
This series makes implementing some of the bdrv_* callbacks easier for block filters by passing requests to bs->file if bs->drv doesn't implement it instead of failing, and adding default bdrv_co_get_block_status() implementations. This is based against Kevin Wolf's block branch, commit

[Qemu-block] [PATCH v5 2/4] block: remove unused bdrv_media_changed

2017-07-13 Thread Manos Pitsidianakis
This function is not used anywhere, so remove it. Markus Armbruster adds: The i82078 floppy device model used to call bdrv_media_changed() to implement its media change bit when backed by a host floppy. This went away in 21fcf36 "fdc: simplify media change handling". Probably broke host floppy

[Qemu-block] [PATCH v5 1/4] block: pass bdrv_* methods to bs->file by default in block filters

2017-07-13 Thread Manos Pitsidianakis
The following functions fail if bs->drv is a filter and does not implement them: bdrv_probe_blocksizes bdrv_probe_geometry bdrv_truncate bdrv_has_zero_init bdrv_get_info Instead, the call should be passed to bs->file if it exists, to allow filter drivers to support those methods without

[Qemu-block] [PATCH v5 3/4] block: remove bdrv_truncate callback in blkdebug

2017-07-13 Thread Manos Pitsidianakis
Now that bdrv_truncate is passed to bs->file by default, remove the callback from block/blkdebug.c and set is_filter to true. Reviewed-by: Eric Blake Reviewed-by: Stefan Hajnoczi Signed-off-by: Manos Pitsidianakis ---

Re: [Qemu-block] [PATCH v5 16/18] qcow2: Switch store_bitmap_data() to byte-based iteration

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
13.07.2017 04:05, Eric Blake wrote: Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by: Eric Blake Reviewed-by: John Snow

Re: [Qemu-block] [PATCH v5 15/18] qcow2: Switch load_bitmap_data() to byte-based iteration

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
13.07.2017 04:05, Eric Blake wrote: Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by: Eric Blake Reviewed-by: John Snow

Re: [Qemu-block] [PATCH 00/18] nbd: BLOCK_STATUS

2017-07-13 Thread Eric Blake
On 02/15/2017 11:05 AM, Paolo Bonzini wrote: [now several months later...] > > > On 03/02/2017 16:47, Vladimir Sementsov-Ogievskiy wrote: >> Hi all! >> >> We really need exporting dirty bitmaps feature as well as remote >> get_block_status for nbd devices. So, here is minimalistic and

Re: [Qemu-block] [PATCH v5 15/18] qcow2: Switch load_bitmap_data() to byte-based iteration

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
13.07.2017 14:03, Vladimir Sementsov-Ogievskiy wrote: 13.07.2017 04:05, Eric Blake wrote: Now that we have adjusted the majority of the calls this function makes to be byte-based, it is easier to read the code if it makes passes over the image using bytes rather than sectors. Signed-off-by:

Re: [Qemu-block] [PATCH v5 1/4] block: pass bdrv_* methods to bs->file by default in block filters

2017-07-13 Thread Eric Blake
On 07/13/2017 05:01 AM, Manos Pitsidianakis wrote: > The following functions fail if bs->drv is a filter and does not > implement them: > > bdrv_probe_blocksizes > bdrv_probe_geometry > bdrv_truncate > bdrv_has_zero_init > bdrv_get_info > > Instead, the call should be passed to bs->file if it

Re: [Qemu-block] [PATCH 5/9] block: List anonymous device BBs in query-block

2017-07-13 Thread Kevin Wolf
Am 12.07.2017 um 17:47 hat Eric Blake geschrieben: > On 07/12/2017 07:57 AM, Kevin Wolf wrote: > > Instead of listing only monitor-owned BlockBackends in query-block, also > > add those anonymous BlockBackends that are owned by a qdev device and as > > such under the control of the user. > > > >

Re: [Qemu-block] [PATCH v5 0/9] Implement NBD_OPT_GO, block size advertisement

2017-07-13 Thread Paolo Bonzini
On 07/07/2017 22:30, Eric Blake wrote: > The NBD protocol has now finalized the extension that fixes > several shortcomings with NBD_OPT_EXPORT_NAME (namely, no error > reporting, no way for the server to advertise block sizes to the > client): >

[Qemu-block] [PATCH v2 0/9] block: -device drive= fixes

2017-07-13 Thread Kevin Wolf
Since 2.9 it is possible for a user to care only about nodes and qdev devices, but not about BlockBackends, by defining with -blockdev and using their node name in -device. Devices create an anonymous BlockBackend internally then. One of the major problems in this setup is that such devices are

[Qemu-block] [PATCH v2 1/9] block: Make blk_get_attached_dev_id() public

2017-07-13 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- block/block-backend.c | 3 +-- include/sysemu/block-backend.h | 1 + 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/block/block-backend.c b/block/block-backend.c index

[Qemu-block] [PATCH v2 3/9] block: Make blk_all_next() public

2017-07-13 Thread Kevin Wolf
Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- block/block-backend.c | 2 +- include/sysemu/block-backend.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/block/block-backend.c b/block/block-backend.c index

[Qemu-block] [PATCH v2 2/9] block/qapi: Add qdev device name to query-block

2017-07-13 Thread Kevin Wolf
With -blockdev/-device, users can indirectly create anonymous BlockBackends, while the state of such backends is still of interest. As a preparation for making such BBs visible in query-block, make sure that they can be identified even without a name by adding the ID/QOM path of their qdev device

Re: [Qemu-block] [Qemu-devel] [PATCH v4 14/17] qcow2: Switch load_bitmap_data() to byte-based iteration

2017-07-13 Thread Eric Blake
On 07/13/2017 04:16 AM, Vladimir Sementsov-Ogievskiy wrote: > 03.07.2017 18:10, Eric Blake wrote: >> Now that we have adjusted the majority of the calls this function >> makes to be byte-based, it is easier to read the code if it makes >> passes over the image using bytes rather than sectors. >>

Re: [Qemu-block] [PATCH v5 0/4] block: Block driver callbacks fixes

2017-07-13 Thread Eric Blake
On 07/13/2017 05:01 AM, Manos Pitsidianakis wrote: > This series makes implementing some of the bdrv_* callbacks easier for block > filters by passing requests to bs->file if bs->drv doesn't implement it > instead > of failing, and adding default bdrv_co_get_block_status() implementations. > >

Re: [Qemu-block] [PATCH v5 1/4] block: pass bdrv_* methods to bs->file by default in block filters

2017-07-13 Thread Manos Pitsidianakis
On Thu, Jul 13, 2017 at 06:23:14AM -0500, Eric Blake wrote: On 07/13/2017 05:01 AM, Manos Pitsidianakis wrote: The following functions fail if bs->drv is a filter and does not implement them: bdrv_probe_blocksizes bdrv_probe_geometry bdrv_truncate bdrv_has_zero_init bdrv_get_info Instead, the

[Qemu-block] [PATCH v2 4/9] block/qapi: Use blk_all_next() for query-block

2017-07-13 Thread Kevin Wolf
This patch replaces the blk_next() loop in query-block by a blk_all_next() one so that we also get access to BlockBackends that aren't owned by the monitor. For now, the next thing we do is check whether each BB has a name, so there is no semantic difference. Signed-off-by: Kevin Wolf

[Qemu-block] [PATCH v2 5/9] block: List anonymous device BBs in query-block

2017-07-13 Thread Kevin Wolf
Instead of listing only monitor-owned BlockBackends in query-block, also add those anonymous BlockBackends that are owned by a qdev device and as such under the control of the user. This allows using query-block to inspect BlockBackends for the modern configuration syntax with -blockdev and

[Qemu-block] [PATCH v2 8/9] qemu-iotests: Test 'info block'

2017-07-13 Thread Kevin Wolf
This test makes sure that all block devices show up on 'info block', with all of the expected information, in different configurations. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/186 | 147 ++

[Qemu-block] [PATCH v2 7/9] scsi-disk: bdrv_attach_dev() for empty CD-ROM

2017-07-13 Thread Kevin Wolf
If no drive=... option is passed (for an empty drive), we don't only lack the BlockBackend normally created by parse_drive(), but we also need to manually call blk_attach_dev(). This fixes at least a segfault when unplugging such devices, the bug that they didn't show up in query-block, and

[Qemu-block] [PATCH v2 9/9] qemu-iotests: Test unplug of -device without drive

2017-07-13 Thread Kevin Wolf
This caused an assertion failure until recently because the BlockBackend would be detached on unplug, but was in fact never attached in the first place. Add a regression test. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake --- tests/qemu-iotests/067

[Qemu-block] [PATCH v2 6/9] ide: bdrv_attach_dev() for empty CD-ROM

2017-07-13 Thread Kevin Wolf
If no drive=... option is passed (for an empty drive), we don't only lack the BlockBackend normally created by parse_drive(), but we also need to manually call blk_attach_dev(). IDE does not support hot unplug, but if it did, qdev would take care to call the matching blk_detach_dev() on unplug.

Re: [Qemu-block] [PATCH v5 1/9] nbd: Create struct for tracking export info

2017-07-13 Thread Vladimir Sementsov-Ogievskiy
07.07.2017 23:30, Eric Blake wrote: The NBD Protocol is introducing some additional information about exports, such as minimum request size and alignment, as well as an advertised maximum request size. It will be easier to feed this information back to the block layer if we gather all the

Re: [Qemu-block] [PATCH] block/vmdk: Report failures in vmdk_read_cid()

2017-07-13 Thread Kevin Wolf
Am 10.07.2017 um 03:58 hat Fam Zheng geschrieben: > On Sun, 07/09 18:06, Peter Maydell wrote: > > The function vmdk_read_cid() can fail if the read on the underlying > > block device fails, or if there's a format error in the VMDK file. > > However its API doesn't provide a mechanism to report

  1   2   >