Re: [Qemu-block] [Qemu-devel] [PATCH] job: drop job_drain

Patchew URL: https://patchew.org/QEMU/20190816170457.522990-1-vsement...@virtuozzo.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ===

[Qemu-block] [PULL 32/36] iotests/257: test traditional sync modes

Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-12-js...@redhat.com [Edit 'Bitmap' --> 'bitmap' in 257.out --js] Signed-off-by: John Snow --- tests/qemu-iotests/257 | 41 +- tests/qemu-iotests/257.out | 3089 2 files

[Qemu-block] [PULL 34/36] block/backup: deal with zero detection

From: Vladimir Sementsov-Ogievskiy We have detect_zeroes option, so at least for blockdev-backup user should define it if zero-detection is needed. For drive-backup leave detection enabled by default but do it through existing option instead of open-coding. Signed-off-by: Vladimir

[Qemu-block] [PULL 35/36] block/backup: refactor write_flags

From: Vladimir Sementsov-Ogievskiy write flags are constant, let's store it in BackupBlockJob instead of recalculating. It also makes two boolean fields to be unused, so, drop them. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: John Snow Reviewed-by: Max Reitz Message-id:

[Qemu-block] [PULL 26/36] iotests/257: test API failures

Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-6-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/257 | 67 ++ tests/qemu-iotests/257.out | 85 ++ 2 files changed, 152

[Qemu-block] [PULL 36/36] tests/test-hbitmap: test next_zero and _next_dirty_area after truncate

From: Vladimir Sementsov-Ogievskiy Test that hbitmap_next_zero and hbitmap_next_dirty_area can find things after old bitmap end. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-id: 20190805164652.42409-1-vsement...@virtuozzo.com Tested-by: John Snow Reviewed-by: John Snow Signed-off-by:

[Qemu-block] [PULL 28/36] block/backup: centralize copy_bitmap initialization

Just a few housekeeping changes that keeps the following commit easier to read; perform the initial copy_bitmap initialization in one place. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-8-js...@redhat.com Signed-off-by: John Snow --- block/backup.c | 29

[Qemu-block] [PULL 29/36] block/backup: add backup_is_cluster_allocated

Modify bdrv_is_unallocated_range to utilize the pnum return from bdrv_is_allocated, and in the process change the semantics from "is unallocated" to "is allocated." Optionally returns a full number of clusters that share the same allocation status. This will be used to carefully toggle bits in

[Qemu-block] [PULL 20/36] qapi: implement block-dirty-bitmap-remove transaction action

It is used to do transactional movement of the bitmap (which is possible in conjunction with merge command). Transactional bitmap movement is needed in scenarios with external snapshot, when we don't want to leave copy of the bitmap in the base image. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PULL 31/36] block/backup: support bitmap sync modes for non-bitmap backups

Accept bitmaps and sync policies for the other backup modes. This allows us to do things like create a bitmap synced to a full backup without a transaction, or start a resumable backup process. Some combinations don't make sense, though: - NEVER policy combined with any non-BITMAP mode doesn't

[Qemu-block] [PULL 33/36] qapi: add dirty-bitmaps to query-named-block-nodes result

From: Vladimir Sementsov-Ogievskiy Let's add a possibility to query dirty-bitmaps not only on root nodes. It is useful when dealing both with snapshots and incremental backups. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: John Snow Message-id:

[Qemu-block] [PULL 17/36] iotests: add test 257 for bitmap-mode backups

Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-18-js...@redhat.com [Removed 'auto' group, as per new testing config guidelines --js] Signed-off-by: John Snow --- tests/qemu-iotests/257 | 416 +++ tests/qemu-iotests/257.out | 2247

[Qemu-block] [PULL 27/36] block/backup: improve sync=bitmap work estimates

When making backups based on bitmaps, the work estimate can be more accurate. Update iotests to reflect the new strategy. TOP work estimates are broken, but do not get worse with this commit. That issue is addressed in the following commits instead. Signed-off-by: John Snow Reviewed-by: Max

[Qemu-block] [PULL 19/36] blockdev: reduce aio_context locked sections in bitmap add/remove

From: Vladimir Sementsov-Ogievskiy Commit 0a6c86d024c52 returned these locks back to add/remove functionality, to protect from intersection of persistent bitmap related IO with other IO. But other bitmap-related functions called here are unrelated to the problem, and there are no needs to keep

[Qemu-block] [PULL 24/36] iotests/257: Refactor backup helpers

This test needs support for non-bitmap backups and missing or unspecified bitmap sync modes, so rewrite the helpers to be a little more generic. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-4-js...@redhat.com Signed-off-by: John Snow ---

[Qemu-block] [PULL 30/36] block/backup: teach TOP to never copy unallocated regions

Presently, If sync=TOP is selected, we mark the entire bitmap as dirty. In the write notifier handler, we dutifully copy out such regions. Fix this in three parts: 1. Mark the bitmap as being initialized before the first yield. 2. After the first yield but before the backup loop, interrogate the

[Qemu-block] [PULL 22/36] iotests/257: add Pattern class

Just kidding, this is easier to manage with a full class instead of a namedtuple. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-2-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/257 | 58 +++--- 1 file

[Qemu-block] [PULL 23/36] iotests/257: add EmulatedBitmap class

Represent a bitmap with an object that we can mark and clear bits in. This makes it easier to manage partial writes when we don't write a full group's worth of patterns before an error. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-3-js...@redhat.com

[Qemu-block] [PULL 21/36] iotests: test bitmap moving inside 254

From: Vladimir Sementsov-Ogievskiy Test persistent bitmap copying with and without removal of original bitmap. Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190708220502.12977-4-js...@redhat.com [Edited comment "bitmap1" -->

[Qemu-block] [PULL 25/36] block/backup: hoist bitmap check into QMP interface

This is nicer to do in the unified QMP interface that we have now, because it lets us use the right terminology back at the user. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190716000117.25219-5-js...@redhat.com Signed-off-by: John Snow --- block/backup.c | 13 -

[Qemu-block] [PULL 14/36] iotests: teach run_job to cancel pending jobs

run_job can cancel pending jobs to simulate failure. This lets us use the pending callback to issue test commands while the job is open, but then still have the job fail in the end. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-15-js...@redhat.com [Maintainer

[Qemu-block] [PULL 16/36] iotests: Add virtio-scsi device helper

Seems that it comes up enough. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-17-js...@redhat.com Signed-off-by: John Snow --- tests/qemu-iotests/040| 6 +- tests/qemu-iotests/093| 6 ++ tests/qemu-iotests/139| 7 ++-

[Qemu-block] [PULL 13/36] iotests: add testing shim for script-style python tests

Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d. Refactor the launcher shim into new and old style shims so that they share environmental configuration. Two cleanup notes: debug was

[Qemu-block] [PULL 05/36] block/backup: Add mirror sync mode 'bitmap'

We don't need or want a new sync mode for simple differences in semantics. Create a new mode simply named "BITMAP" that is designed to make use of the new Bitmap Sync Mode field. Because the only bitmap sync mode is 'on-success', this adds no new functionality to the backup job (yet). The old

[Qemu-block] [PULL 15/36] iotests: teach FilePath to produce multiple paths

Use "FilePaths" instead of "FilePath" to request multiple files be cleaned up after we leave that object's scope. This is not crucial; but it saves a little typing. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-16-js...@redhat.com Signed-off-by: John Snow

[Qemu-block] [PULL 18/36] block/backup: loosen restriction on readonly bitmaps

With the "never" sync policy, we actually can utilize readonly bitmaps now. Loosen the check at the QMP level, and tighten it based on provided arguments down at the job creation level instead. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-19-js...@redhat.com

[Qemu-block] [PULL 11/36] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap

This simplifies some interface matters; namely the initialization and (later) merging the manifest back into the sync_bitmap if it was provided. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-12-js...@redhat.com Signed-off-by: John Snow --- block/backup.c |

[Qemu-block] [PULL 12/36] block/backup: add 'always' bitmap sync policy

This adds an "always" policy for bitmap synchronization. Regardless of if the job succeeds or fails, the bitmap is *always* synchronized. This means that for backups that fail part-way through, the bitmap retains a record of which sectors need to be copied out to accomplish a new backup using the

[Qemu-block] [PULL 10/36] block/dirty-bitmap: add bdrv_dirty_bitmap_get

Add a public interface for get. While we're at it, rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked". (There are more functions to rename to the bdrv_dirty_bitmap_VERB form, but they will wait until the conclusion of this series.) Signed-off-by: John Snow Reviewed-by: Max

[Qemu-block] [PULL 09/36] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal

I'm surprised it didn't come up sooner, but sometimes we have a +busy bitmap as a source. This is dangerous from the QMP API, but if we are the owner that marked the bitmap busy, it's safe to merge it using it as a read only source. It is not safe in the general case to allow users to read from

[Qemu-block] [PULL 04/36] qapi: add BitmapSyncMode enum

Depending on what a user is trying to accomplish, there might be a few bitmap cleanup actions that occur when an operation is finished that could be useful. I am proposing three: - NEVER: The bitmap is never synchronized against what was copied. - ALWAYS: The bitmap is always synchronized, even

[Qemu-block] [PULL 08/36] hbitmap: enable merging across granularities

Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-9-js...@redhat.com Signed-off-by: John Snow --- util/hbitmap.c | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index

[Qemu-block] [PULL 06/36] block/backup: add 'never' policy to bitmap sync mode

This adds a "never" policy for bitmap synchronization. Regardless of if the job succeeds or fails, we never update the bitmap. This can be used to perform differential backups, or simply to avoid the job modifying a bitmap. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id:

[Qemu-block] [PULL 01/36] qapi/block-core: Introduce BackupCommon

drive-backup and blockdev-backup have an awful lot of things in common that are the same. Let's fix that. I don't deduplicate 'target', because the semantics actually did change between each structure. Leave that one alone so it can be documented separately. Where documentation was not

[Qemu-block] [PULL 03/36] blockdev-backup: utilize do_backup_common

Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-4-js...@redhat.com Signed-off-by: John Snow --- blockdev.c | 65 +- 1 file changed, 6 insertions(+), 59 deletions(-) diff --git a/blockdev.c b/blockdev.c index

[Qemu-block] [PULL 07/36] hbitmap: Fix merge when b is empty, and result is not an alias of a

Nobody calls the function like this currently, but we neither prohibit or cope with this behavior. I decided to make the function cope with it. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-8-js...@redhat.com Signed-off-by: John Snow --- util/hbitmap.c | 13

[Qemu-block] [PULL 00/36] Bitmaps patches

The following changes since commit afd760539308a5524accf964107cdb1d54a059e3: Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190816' into staging (2019-08-16 17:21:40 +0100) are available in the Git repository at: https://github.com/jnsnow/qemu.git tags/bitmaps-pull

[Qemu-block] [PULL 02/36] drive-backup: create do_backup_common

Create a common core that comprises the actual meat of what the backup API boundary needs to do, and then switch drive-backup to use it. Signed-off-by: John Snow Reviewed-by: Max Reitz Message-id: 20190709232550.10724-3-js...@redhat.com Signed-off-by: John Snow --- blockdev.c | 122

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On 8/16/19 6:45 PM, Nir Soffer wrote: > On Sat, Aug 17, 2019 at 12:57 AM John Snow > wrote: > > On 8/16/19 5:21 PM, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On Sat, Aug 17, 2019 at 12:57 AM John Snow wrote: > On 8/16/19 5:21 PM, Nir Soffer wrote: > > When creating an image with preallocation "off" or "falloc", the first > > block of the image is typically not allocated. When using Gluster > > storage backed by XFS filesystem, reading this block

Re: [Qemu-block] bitmaps branch conflict resolution

On 17.08.19 00:07, John Snow wrote: > Hi Max, I took your patch and adjusted it slightly: I don't like > "skip_bytes" anymore because it's clear now that we don't only read that > value when we're skipping bytes, so now it's just status_bytes. Yep, sure. > Since this is based on your fixup,

[Qemu-block] bitmaps branch conflict resolution

Hi Max, I took your patch and adjusted it slightly: I don't like "skip_bytes" anymore because it's clear now that we don't only read that value when we're skipping bytes, so now it's just status_bytes. Since this is based on your fixup, would you like to offer an Ack/S-o-b/R-B/whichever here?

Re: [Qemu-block] [PATCH] block: posix: Always allocate the first block

On 8/16/19 5:21 PM, Nir Soffer wrote: > When creating an image with preallocation "off" or "falloc", the first > block of the image is typically not allocated. When using Gluster > storage backed by XFS filesystem, reading this block using direct I/O > succeeds regardless of request length,

[Qemu-block] [PATCH] block: posix: Always allocate the first block

When creating an image with preallocation "off" or "falloc", the first block of the image is typically not allocated. When using Gluster storage backed by XFS filesystem, reading this block using direct I/O succeeds regardless of request length, fooling alignment detection. In this case we

Re: [Qemu-block] [PATCH v5 3/6] iotests: Add casenotrun report to bash tests

On Thu, Aug 15, 2019 at 08:44:11PM -0400, John Snow wrote: > > > On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > > The new function _casenotrun() is to be invoked if a test case cannot > > be run for some reason. The user will be notified by a message passed > > to the function. > > > > Oh, I

Re: [Qemu-block] [Qemu-devel] [PATCH v5 0/6] Allow Valgrind checking all QEMU processes

On Fri, Jul 19, 2019 at 07:30:10PM +0300, Andrey Shinkevich wrote: > In the current implementation of the QEMU bash iotests, only qemu-io > processes may be run under the Valgrind, which is a useful tool for > finding memory usage issues. Let's allow the common.rc bash script > runing all the QEMU

Re: [Qemu-block] [PATCH 0/4] backup: fix skipping unallocated clusters

On 8/14/19 12:54 PM, Vladimir Sementsov-Ogievskiy wrote: > > > 14 авг. 2019 г. 17:43 пользователь Vladimir Sementsov-Ogievskiy > написал: > > Hi all! > > There is a bug in not yet merged patch > "block/backup: teach TOP to never copy unallocated regions" > in

Re: [Qemu-block] [PATCH] virtio-blk: Cancel the pending BH when the dataplane is reset

On 8/16/19 1:15 PM, Philippe Mathieu-Daudé wrote: > When 'system_reset' is called, the main loop clear the memory > region cache before the BH has a chance to execute. Later when > the deferred function is called, some assumptions that were > made when scheduling them are no longer true when

Re: [Qemu-block] [PATCH for-4.2 09/13] qcow2: Fix overly long snapshot tables

On 31.07.19 11:22, Max Reitz wrote: > On 30.07.19 21:08, Eric Blake wrote: >> On 7/30/19 12:25 PM, Max Reitz wrote: >>> We currently refuse to open qcow2 images with overly long snapshot >>> tables. This patch makes qemu-img check -r all drop all offending >>> entries past what we deem

[Qemu-block] [PATCH] virtio-blk: Cancel the pending BH when the dataplane is reset

When 'system_reset' is called, the main loop clear the memory region cache before the BH has a chance to execute. Later when the deferred function is called, some assumptions that were made when scheduling them are no longer true when they actually execute. This is what happens using a virtio-blk

Re: [Qemu-block] [PATCH] job: drop job_drain

16.08.2019 20:10, Vladimir Sementsov-Ogievskiy wrote: > 16.08.2019 20:04, Vladimir Sementsov-Ogievskiy wrote: >> In job_finish_sync job_enter should be enough for a job to make some >> progress and draining is a wrong tool for it. So use job_enter directly >> here and drop job_drain with all

Re: [Qemu-block] [PATCH] job: drop job_drain

16.08.2019 20:04, Vladimir Sementsov-Ogievskiy wrote: > In job_finish_sync job_enter should be enough for a job to make some > progress and draining is a wrong tool for it. So use job_enter directly > here and drop job_drain with all related staff not used more. > > Suggested-by: Kevin Wolf >

[Qemu-block] [PATCH] job: drop job_drain

In job_finish_sync job_enter should be enough for a job to make some progress and draining is a wrong tool for it. So use job_enter directly here and drop job_drain with all related staff not used more. Suggested-by: Kevin Wolf Signed-off-by: Vladimir Sementsov-Ogievskiy --- It's a

Re: [Qemu-block] [PATCH v3 0/4] qcow2: async handling of fragmented io

15.08.2019 18:39, Vladimir Sementsov-Ogievskiy wrote: > 15.08.2019 17:09, Max Reitz wrote: >> On 15.08.19 14:10, Vladimir Sementsov-Ogievskiy wrote: >>> Hi all! >>> >>> Here is an asynchronous scheme for handling fragmented qcow2 >>> reads and writes. Both qcow2 read and write functions loops

Re: [Qemu-block] [Qemu-devel] [PULL 00/16] Block layer patches

On Fri, 16 Aug 2019 at 10:36, Kevin Wolf wrote: > > The following changes since commit 9e06029aea3b2eca1d5261352e695edc1e7d7b8b: > > Update version for v4.1.0 release (2019-08-15 13:03:37 +0100) > > are available in the Git repository at: > > git://repo.or.cz/qemu/kevin.git tags/for-upstream

Re: [Qemu-block] [PATCH v6 37/42] block: Leave BDS.backing_file constant

09.08.2019 19:14, Max Reitz wrote: > Parts of the block layer treat BDS.backing_file as if it were whatever > the image header says (i.e., if it is a relative path, it is relative to > the overlay), other parts treat it like a cache for > bs->backing->bs->filename (relative paths are relative to

[Qemu-block] [PATCH v4 1/5] tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache

qemu-io now prefixes its error and warnings with "qemu-io:". 36b9986b08787019e fixed a lot of iotests output but forget about 026.out.nocache. Fix it too. Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()") Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH v4 5/5] block/qcow2: introduce parallel subrequest handling in read and write

It improves performance for fragmented qcow2 images. It also affect 026 iotest, increasing leaked clusters number, which is not surprising when we run several sub-requests of qcow2 request in parallel. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/qcow2.h | 3 +

[Qemu-block] [PATCH v4 4/5] block/qcow2: refactor qcow2_co_pwritev_part

Similarly to previous commit, prepare for parallelizing write-loop iterations. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- block/qcow2.c | 153 +- 1 file changed, 89 insertions(+), 64 deletions(-) diff --git

[Qemu-block] [PATCH v4 3/5] block/qcow2: refactor qcow2_co_preadv_part

Further patch will run partial requests of iterations of qcow2_co_preadv in parallel for performance reasons. To prepare for this, separate part which may be parallelized into separate function (qcow2_co_preadv_task). While being here, also separate encrypted clusters reading to own function,

[Qemu-block] [PATCH v4 0/5] qcow2: async handling of fragmented io

Hi all! Here is an asynchronous scheme for handling fragmented qcow2 reads and writes. Both qcow2 read and write functions loops through sequential portions of data. The series aim it to parallelize these loops iterations. It improves performance for fragmented qcow2 images, I've tested it as

[Qemu-block] [PATCH v4 2/5] block: introduce aio task pool

Common interface for aio task loops. To be used for improving performance of synchronous io loops in qcow2, block-stream, copy-on-read, and may be other places. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- include/block/aio_task.h | 54 +

Re: [Qemu-block] [PATCH v6 35/42] block: Fix check_to_replace_node()

16.08.2019 16:30, Max Reitz wrote: > On 16.08.19 13:01, Vladimir Sementsov-Ogievskiy wrote: >> 15.08.2019 20:01, Max Reitz wrote: >>> On 15.08.19 17:21, Vladimir Sementsov-Ogievskiy wrote: 09.08.2019 19:14, Max Reitz wrote: > Currently, check_to_replace_node() only allows mirror to

Re: [Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

On Fri 16 Aug 2019 04:08:19 PM CEST, Kevin Wolf wrote: >> And yes, the odd value on the 512KB row on that we discussed last month >> is due to this same bug: >> >> https://lists.gnu.org/archive/html/qemu-block/2019-07/msg00496.html > > Hm... And suddently it makes sense. :-) > > So I assume all

Re: [Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

Am 16.08.2019 um 15:30 hat Alberto Garcia geschrieben: > On Fri 16 Aug 2019 02:59:21 PM CEST, Kevin Wolf wrote: > > The requirement so that this bug doesn't affect the user seems to be > > that the image size is a multiple of 64k * 8k = 512 MB. Which means > > that users are probably often lucky

Re: [Qemu-block] [PATCH v6 35/42] block: Fix check_to_replace_node()

On 16.08.19 13:01, Vladimir Sementsov-Ogievskiy wrote: > 15.08.2019 20:01, Max Reitz wrote: >> On 15.08.19 17:21, Vladimir Sementsov-Ogievskiy wrote: >>> 09.08.2019 19:14, Max Reitz wrote: Currently, check_to_replace_node() only allows mirror to replace a node in the chain of the source

Re: [Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

On Fri 16 Aug 2019 02:59:21 PM CEST, Kevin Wolf wrote: > The requirement so that this bug doesn't affect the user seems to be > that the image size is a multiple of 64k * 8k = 512 MB. Which means > that users are probably often lucky enough in practice. Or rather: cluster_size^2 / 8, which, if my

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

16.08.2019 15:33, Markus Armbruster wrote: > Kevin Wolf writes: > >> Am 15.08.2019 um 21:24 hat Markus Armbruster geschrieben: > [...] >>> Let's assume all libvirt ever does with deprecation notices is logging >>> them. Would that solve the problem of reliably alerting libvirt >>> developers to

Re: [Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

Am 16.08.2019 um 14:17 hat Alberto Garcia geschrieben: > The size of the qcow2 L2 cache defaults to 32 MB, which can be easily > larger than the maximum amount of L2 metadata that the image can have. > For example: with 64 KB clusters the user would need a qcow2 image > with a virtual size of 256

Re: [Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

Cc qemu-stable This bug means that under certain conditions it's impossible to create a cache large enough for the image, resulting in reduced I/O performance. On Fri, Aug 16, 2019 at 03:17:42PM +0300, Alberto Garcia wrote: > The size of the qcow2 L2 cache defaults to 32 MB, which can be easily

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] qapi: deprecate implicit filters

Kevin Wolf writes: > Am 15.08.2019 um 21:24 hat Markus Armbruster geschrieben: [...] >> Let's assume all libvirt ever does with deprecation notices is logging >> them. Would that solve the problem of reliably alerting libvirt >> developers to deprecation issues? Nope. But it could help >>

[Qemu-block] [PATCH] qcow2: Fix the calculation of the maximum L2 cache size

The size of the qcow2 L2 cache defaults to 32 MB, which can be easily larger than the maximum amount of L2 metadata that the image can have. For example: with 64 KB clusters the user would need a qcow2 image with a virtual size of 256 GB in order to have 32 MB of L2 metadata. Because of that,

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/42] Invert Endian bit in SPARCv9 MMU TTE

On Fri, Aug 16, 2019 at 11:58:05AM +0200, Philippe Mathieu-Daudé wrote: > Hi Tony, > > On 8/16/19 8:28 AM, tony.ngu...@bt.com wrote: > > This patchset implements the IE (Invert Endian) bit in SPARCv9 MMU TTE. > > > > v7: > [...] > > - Re-declared many native endian devices as little or big

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/42] Invert Endian bit in SPARCv9 MMU TTE

On Fri, 16 Aug 2019 at 12:37, wrote: > > Hi Phillippe, > > On 8/16/19 7:58 PM, Philippe Mathieu-Daudé wrote: > >On 8/16/19 8:28 AM, tony.ngu...@bt.com wrote: > >> For each device declared with DEVICE_NATIVE_ENDIAN, find the set of > >> targets from the set of target/hw/*/device.o. > >> > >> If

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/42] Invert Endian bit in SPARCv9 MMU TTE

Hi Phillippe, On 8/16/19 7:58 PM, Philippe Mathieu-Daudé wrote: >On 8/16/19 8:28 AM, tony.ngu...@bt.com wrote: >> This patchset implements the IE (Invert Endian) bit in SPARCv9 MMU TTE. >> >> v7: >[...] >> - Re-declared many native endian devices as little or big endian. This is why >> v7 has

Re: [Qemu-block] [PATCH v4] blockjob: drain all job nodes in block_job_drain

15.08.2019 16:15, Max Reitz wrote: > On 02.08.19 11:52, Vladimir Sementsov-Ogievskiy wrote: >> Instead of draining additional nodes in each job code, let's do it in >> common block_job_drain, draining just all job's children. >> BlockJobDriver.drain becomes unused, so, drop it at all. >> >> It's

Re: [Qemu-block] [PATCH v6 35/42] block: Fix check_to_replace_node()

15.08.2019 20:01, Max Reitz wrote: > On 15.08.19 17:21, Vladimir Sementsov-Ogievskiy wrote: >> 09.08.2019 19:14, Max Reitz wrote: >>> Currently, check_to_replace_node() only allows mirror to replace a node >>> in the chain of the source node, and only if it is the first non-filter >>> node below

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

16.08.2019 13:23, Vladimir Sementsov-Ogievskiy wrote: > 15.08.2019 21:50, Eric Blake wrote: >> The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be >> advertised when the server promises cache consistency between >> simultaneous clients (basically, rules that determine what FUA and

Re: [Qemu-block] [PATCH] nbd: Advertise multi-conn for shared read-only connections

15.08.2019 21:50, Eric Blake wrote: > The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be > advertised when the server promises cache consistency between > simultaneous clients (basically, rules that determine what FUA and > flush from one client are able to guarantee for reads

Re: [Qemu-block] [Qemu-devel] [PULL 00/16] Block layer patches

Patchew URL: https://patchew.org/QEMU/20190816093439.14262-1-kw...@redhat.com/ Hi, This series failed build test on s390x host. Please find the details below. === TEST SCRIPT BEGIN === #!/bin/bash # Testing script will be invoked under the git checkout with # HEAD pointing to a commit that

Re: [Qemu-block] [qemu-s390x] [Qemu-devel] [PATCH v7 33/42] exec: Replace device_endian with MemOp

On 8/16/19 9:37 AM, tony.ngu...@bt.com wrote: > Simplify endianness comparisons with consistent use of the more > expressive MemOp. > > Suggested-by: Richard Henderson > Signed-off-by: Tony Nguyen > Reviewed-by: Richard Henderson > Acked-by: David Gibson This patch is *huge*, more than

Re: [Qemu-block] [Qemu-devel] [PATCH v7 27/42] hw/pci-host: Declare device little or big endian

On 8/16/19 9:35 AM, tony.ngu...@bt.com wrote: > For each device declared with DEVICE_NATIVE_ENDIAN, find the set of > targets from the set of target/hw/*/device.o. > > If the set of targets are all little or all big endian, re-declare > the device endianness as DEVICE_LITTLE_ENDIAN or

Re: [Qemu-block] [Qemu-devel] [PATCH v7 25/42] hw/misc: Declare device little or big endian

On 8/16/19 9:34 AM, tony.ngu...@bt.com wrote: > For each device declared with DEVICE_NATIVE_ENDIAN, find the set of > targets from the set of target/hw/*/device.o. > > If the set of targets are all little or all big endian, re-declare > the device endianness as DEVICE_LITTLE_ENDIAN or

Re: [Qemu-block] [Qemu-devel] [PATCH v7 24/42] hw/isa: Declare device little or big endian

On 8/16/19 9:34 AM, tony.ngu...@bt.com wrote: > For each device declared with DEVICE_NATIVE_ENDIAN, find the set of > targets from the set of target/hw/*/device.o. > > If the set of targets are all little or all big endian, re-declare > the device endianness as DEVICE_LITTLE_ENDIAN or

Re: [Qemu-block] [Qemu-devel] [PATCH v7 00/42] Invert Endian bit in SPARCv9 MMU TTE

Hi Tony, On 8/16/19 8:28 AM, tony.ngu...@bt.com wrote: > This patchset implements the IE (Invert Endian) bit in SPARCv9 MMU TTE. > > v7: [...] > - Re-declared many native endian devices as little or big endian. This is why > v7 has +16 patches. Why are you doing that? What is the rational?

[Qemu-block] [PATCH] file-posix: Fix has_write_zeroes after NO_FALLBACK

If QEMU_AIO_NO_FALLBACK is given, we always return failure and don't even try to use the BLKZEROOUT ioctl. In this failure case, we shouldn't disable has_write_zeroes because we didn't learn anything about the ioctl. The next request might not set QEMU_AIO_NO_FALLBACK and we can still use the

[Qemu-block] [PULL 12/16] block: Remove blk_pread_unthrottled()

The functionality offered by blk_pread_unthrottled() goes back to commit 498e386c584. Then, we couldn't perform I/O throttling with synchronous requests because timers wouldn't be executed in polling loops. So the commit automatically disabled I/O throttling as soon as a synchronous request was

[Qemu-block] [PULL 11/16] iotests: Add test for concurrent stream/commit

From: Max Reitz We already have 030 for that in general, but this tests very specific cases of both jobs finishing concurrently. Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf --- tests/qemu-iotests/258 | 163 + tests/qemu-iotests/258.out | 33

[Qemu-block] [PULL 14/16] block-backend: Queue requests while drained

This fixes devices like IDE that can still start new requests from I/O handlers in the CPU thread while the block backend is drained. The basic assumption is that in a drain section, no new requests should be allowed through a BlockBackend (blk_drained_begin/end don't exist, we get drain sections

[Qemu-block] [PULL 16/16] file-posix: Handle undetectable alignment

From: Nir Soffer In some cases buf_align or request_alignment cannot be detected: 1. With Gluster, buf_align cannot be detected since the actual I/O is done on Gluster server, and qemu buffer alignment does not matter. Since we don't have alignment requirement, buf_align=1 is the best

[Qemu-block] [PULL 15/16] qemu-img convert: Deprecate using -n and -o together

bdrv_create options specified with -o have no effect when skipping image creation with -n, so this doesn't make sense. Warn against the misuse and deprecate the combination so we can make it a hard error later. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz Reviewed-by: John Snow

[Qemu-block] [PULL 13/16] mirror: Keep mirror_top_bs drained after dropping permissions

mirror_top_bs is currently implicitly drained through its connection to the source or the target node. However, the drain section for target_bs ends early after moving mirror_top_bs from src to target_bs, so that requests can already be restarted while mirror_top_bs is still present in the chain,

[Qemu-block] [PULL 09/16] tests: Test polling in bdrv_drop_intermediate()

From: Max Reitz Signed-off-by: Max Reitz Signed-off-by: Kevin Wolf --- tests/test-bdrv-drain.c | 167 1 file changed, 167 insertions(+) diff --git a/tests/test-bdrv-drain.c b/tests/test-bdrv-drain.c index 03fa1142a1..1600d41e9a 100644 ---

[Qemu-block] [PULL 08/16] block: Reduce (un)drains when replacing a child

From: Max Reitz Currently, bdrv_replace_child_noperm() undrains the parent until it is completely undrained, then re-drains it after attaching the new child node. This is a problem with bdrv_drop_intermediate(): We want to keep the whole subtree drained, including parents, while the operation

[Qemu-block] [PULL 10/16] tests: Test mid-drain bdrv_replace_child_noperm()

From: Max Reitz Add a test for what happens when you call bdrv_replace_child_noperm() for various drain situations ({old,new} child {drained,not drained}). Most importantly, if both the old and the new child are drained, the parent must not be undrained at any point. Signed-off-by: Max Reitz

[Qemu-block] [PULL 04/16] iotests: Move migration helpers to iotests.py

234 implements functions that are useful for doing migration between two VMs. Move them to iotests.py so that other test cases can use them, too. Signed-off-by: Kevin Wolf Reviewed-by: Max Reitz --- tests/qemu-iotests/234| 30 +++---

[Qemu-block] [PULL 05/16] iotests: Test migration with all kinds of filter nodes

This test case is motivated by commit 2b23f28639 ('block/copy-on-read: Fix permissions for inactive node'). Instead of just testing copy-on-read on migration, let's stack all sorts of filter nodes on top of each other and try if the resulting VM can still migrate successfully. For good measure,

[Qemu-block] [PULL 06/16] block: Simplify bdrv_filter_default_perms()

The same change as commit 2b23f28639 ('block/copy-on-read: Fix permissions for inactive node') made for the copy-on-read driver can be made for bdrv_filter_default_perms(): Retaining the old permissions from the BdrvChild if it is given complicates things unnecessarily when in the end this only

[Qemu-block] [PULL 07/16] block: Keep subtree drained in drop_intermediate

From: Max Reitz bdrv_drop_intermediate() calls BdrvChildRole.update_filename(). That may poll, thus changing the graph, which potentially breaks the QLIST_FOREACH_SAFE() loop. Just keep the whole subtree drained. This is probably the right thing to do anyway (dropping nodes while the subtree

[Qemu-block] [PULL 02/16] iotests/118: Create test classes dynamically

We're getting a ridiculous number of child classes of TestInitiallyFilled and TestInitiallyEmpty that differ only in a few attributes that we want to test in all combinations. Instead of explicitly writing down every combination, let's use a loop and create those classes dynamically.

  1   2   >