Re: [Qemu-block] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-28 Thread Vladimir Sementsov-Ogievskiy
27.08.2019 23:12, John Snow wrote: > > > On 8/23/19 5:22 AM, Vladimir Sementsov-Ogievskiy wrote: >> 14.08.2019 13:07, Vladimir Sementsov-Ogievskiy wrote: >>> To get rid of implicit filters related workarounds in future let's >>> deprecate them now. >> >> Interesting, could we deprecate implicit

Re: [Qemu-block] [PATCH v5 1/4] block: Add zoned device model property

2019-08-28 Thread Stefan Hajnoczi
On Fri, Aug 23, 2019 at 03:49:24PM -0400, Dmitry Fomichev wrote: > +uint8_t bdrv_is_zoned(BlockDriverState *bs) > +{ > +/* > + * Host Aware zone devices are supposed to be able to work > + * just like regular block devices. Thus, we only consider > + * Host Managed devices to be

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] vhost-user-blk: prevent using uninitialized vqs

2019-08-28 Thread Stefan Hajnoczi
On Thu, Aug 22, 2019 at 11:34:24AM -0700, Raphael Norwitz wrote: > Same rational as: e6cc11d64fc998c11a4dfcde8fda3fc33a74d844 > > Of the 3 virtqueues, seabios only sets cmd, leaving ctrl > and event without a physical address. This can cause > vhost_verify_ring_part_mapping to return ENOMEM,

Re: [Qemu-block] [Qemu-devel] [PATCH 1/2] block/nvme: add support for write zeros

2019-08-28 Thread Maxim Levitsky
On Tue, 2019-08-27 at 18:22 -0400, John Snow wrote: > Without a commit message, I have no real hope of reviewing this. I was > CC'd, though, so I'll give it a blind shot. > > We want to add write_zeroes support for block/nvme, but I can't really > verify any of that is correct or working without

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/nvme: add support for discard

2019-08-28 Thread Maxim Levitsky
On Tue, 2019-08-27 at 18:29 -0400, John Snow wrote: > > On 8/25/19 3:15 AM, Maxim Levitsky wrote: > > Signed-off-by: Maxim Levitsky > > --- > > block/nvme.c | 83 ++ > > block/trace-events | 2 ++ > > 2 files changed, 85 insertions(+) > > > >

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

2019-08-28 Thread Vladimir Sementsov-Ogievskiy
23.08.2019 17:34, Eric Blake wrote: > While it may be counterintuitive at first, the introduction of > NBD_CMD_WRITE_ZEROES and NBD_CMD_BLOCK_STATUS has caused a performance > regression in qemu [1], when copying a sparse file. When the > destination file must contain the same contents as the

Re: [Qemu-block] [PATCH v2 2/2] block: Remove unused masks

2019-08-28 Thread Juan Quintela
Nir Soffer wrote: > Replace confusing usage: > > ~BDRV_SECTOR_MASK > > With more clear: > > (BDRV_SECTOR_SIZE - 1) > > Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was > it's last user. > > Signed-off-by: Nir Soffer Reviewed-by: Juan Quintela

Re: [Qemu-block] [Qemu-devel] [PATCH v5 4/4] raw: Don't open ZBDs if backend can't handle them

2019-08-28 Thread Stefan Hajnoczi
On Fri, Aug 23, 2019 at 03:49:27PM -0400, Dmitry Fomichev wrote: > diff --git a/block/file-posix.c b/block/file-posix.c > index d9f2fc5e46..090e7c4d2f 100644 > --- a/block/file-posix.c > +++ b/block/file-posix.c > @@ -2884,6 +2884,20 @@ static int raw_check_perm(BlockDriverState *bs, > uint64_t

Re: [Qemu-block] [Qemu-devel] [PATCH v4 0/3] qcow2: add zstd cluster compression

2019-08-28 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190828125654.10544-1-dplotni...@virtuozzo.com/ Hi, This series seems to have some coding style problems. See output below for more information: Message-id: 20190828125654.10544-1-dplotni...@virtuozzo.com Type: series Subject: [Qemu-devel] [PATCH v4

[Qemu-block] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

2019-08-28 Thread Thomas Huth
In our documentation, we use a mix of "$QEMU", "qemu-system-i386" and "qemu-system-x86_64" when we give examples to the users how to run QEMU. Some more consistency would be good here. Also some distributions use different names for the QEMU binary (e.g. "qemu-kvm" in RHEL), so providing more

Re: [Qemu-block] [PATCH v5 3/4] block/ide/scsi: Set BLK_PERM_SUPPORT_ZONED

2019-08-28 Thread Stefan Hajnoczi
On Fri, Aug 23, 2019 at 03:49:26PM -0400, Dmitry Fomichev wrote: > Added a new boolean argument to blkconf_apply_backend_options() > to let the common block code know whether the chosen block > backend can handle zoned block devices or not. > > blkconf_apply_backend_options() then sets

[Qemu-block] [PATCH v4 2/3] qcow2: rework the cluster compression routine

2019-08-28 Thread Denis Plotnikov
The patch allow to process image compression type defined in the image header and choose an appropriate method for image clusters (de)compression. Signed-off-by: Denis Plotnikov --- block/qcow2-threads.c | 78 +++ 1 file changed, 64 insertions(+), 14

Re: [Qemu-block] [PATCH 0/2] block/file-posix: Reduce xfsctl() use

2019-08-28 Thread Stefano Garzarella
On Fri, Aug 23, 2019 at 03:03:39PM +0200, Max Reitz wrote: > Hi, > > As suggested by Paolo, this series drops xfsctl() calls where we have > working fallocate() alternatives. (And thus replaces “block/file-posix: > Fix xfs_write_zeroes()”.) > > Unfortunately, we also use xfsctl() to inquire the

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

2019-08-28 Thread Eric Blake
On 8/28/19 4:57 AM, Vladimir Sementsov-Ogievskiy wrote: >> Hence, it is desirable to have a way for clients to specify that a >> particular write zero request is being attempted for a fast wipe, and >> get an immediate failure if the zero request would otherwise take the >> same time as a write.

[Qemu-block] [PATCH v4 1/3] qcow2: introduce compression type feature

2019-08-28 Thread Denis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to QCOW2 header that indicates that *all* compressed clusters must be (de)compressed using a certain compression type. It is implied that the compression type is set on the image creation and can be changed only later

[Qemu-block] [PATCH v4 0/3] qcow2: add zstd cluster compression

2019-08-28 Thread Denis Plotnikov
v4: * remove not feasible switch case [Vladimir] * add sanity checks to zstd decompresssion [Vladimir] * store zstd compressed length in big endian [Max, Kevin] v3: * relax the compression type setting requirement when the compression type is not zlib [Eric, Kevin] * add compression type values

Re: [Qemu-block] [PATCH v5 0/4] virtio/block: handle zoned backing devices

2019-08-28 Thread Stefan Hajnoczi
On Fri, Aug 23, 2019 at 03:49:23PM -0400, Dmitry Fomichev wrote: > Dmitry Fomichev (4): > block: Add zoned device model property > raw: Recognize zoned backing devices > block/ide/scsi: Set BLK_PERM_SUPPORT_ZONED > raw: Don't open ZBDs if backend can't handle them The overall approach

Re: [Qemu-block] [PATCH v5 2/4] raw: Recognize zoned backing devices

2019-08-28 Thread Stefan Hajnoczi
On Fri, Aug 23, 2019 at 03:49:25PM -0400, Dmitry Fomichev wrote: > +static int hdev_get_zoned_model(int fd) Please use the enum: static BdrvZonedModel hdev_get_zoned_model(int fd) > +{ > +#ifdef CONFIG_LINUX > +char buf[32]; > +int ret; > + > +ret = hdev_read_blk_queue_entry(fd,

[Qemu-block] [PATCH v4 3/3] qcow2: add zstd cluster compression

2019-08-28 Thread Denis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of compression ratio in comparison with zlib, which, at the moment, has been the only compression method available. The performance test results: Test compresses and

Re: [Qemu-block] [PATCH v5 6/6] iotests: extend sleeping time under Valgrind

2019-08-28 Thread Andrey Shinkevich
On 27/08/2019 22:42, John Snow wrote: > > > On 8/23/19 11:27 AM, Vladimir Sementsov-Ogievskiy wrote: >> 16.08.2019 4:01, John Snow wrote: >>> >>> >>> On 7/19/19 12:30 PM, Andrey Shinkevich wrote: To synchronize the time when QEMU is running longer under the Valgrind, increase the

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

2019-08-28 Thread Vladimir Sementsov-Ogievskiy
28.08.2019 16:04, Eric Blake wrote: > On 8/28/19 4:57 AM, Vladimir Sementsov-Ogievskiy wrote: > >>> Hence, it is desirable to have a way for clients to specify that a >>> particular write zero request is being attempted for a fast wipe, and >>> get an immediate failure if the zero request would

Re: [Qemu-block] [PATCH v9 02/13] block/backup: split shareable copying part from backup_do_cow

2019-08-28 Thread Max Reitz
On 28.08.19 16:27, Vladimir Sementsov-Ogievskiy wrote: > 28.08.2019 17:22, Max Reitz wrote: >> On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: >>> Split copying logic which will be shared with backup-top filter. >>> >>> Signed-off-by: Vladimir Sementsov-Ogievskiy >>> --- >>>

Re: [Qemu-block] [PATCH v5 1/6] iotests: allow Valgrind checking all QEMU processes

2019-08-28 Thread Andrey Shinkevich
On 27/08/2019 22:56, John Snow wrote: > > > On 8/25/19 11:26 AM, Andrey Shinkevich wrote: >> >> >> On 16/08/2019 01:49, John Snow wrote: >>> >>> >>> On 7/19/19 12:30 PM, Andrey Shinkevich wrote: With the '-valgrind' option, let all the QEMU processes be run under the Valgrind tool.

Re: [Qemu-block] [PATCH v9 04/13] block/backup: adjust block-copy functions style

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Fix comment style and reflow arguments in same manner like > block_copy_state_new. I like the current function header style better. Max > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/backup.c | 40

Re: [Qemu-block] [PATCH v9 05/13] block: move block_copy from block/backup.c to separate file

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Split block_copy to separate file, to be cleanly shared with backup-top > filter driver in further commits. > > It's a clean movement, the only change is drop "static" from interface > functions. > > Signed-off-by: Vladimir

Re: [Qemu-block] [PATCH v9 03/13] block/backup: introduce BlockCopyState

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Split copying code part from backup to "block-copy", including separate > state structure and function renaming. This is needed to share it with > backup-top filter driver in further commits. > > Notes: > > 1. As BlockCopyState keeps own

Re: [Qemu-block] [PATCH v9 06/13] block: teach bdrv_debug_breakpoint skip filters with backing

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Teach bdrv_debug_breakpoint and bdrv_debug_remove_breakpoint skip > filters with backing. This is needed to implement and use in backup job > it's own backup_top filter driver (like mirror already has one), and > without this improvement,

Re: [Qemu-block] [PATCH v9 01/13] block/backup: fix backup_cow_with_offload for last cluster

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > We shouldn't try to copy bytes beyond EOF. Fix it. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/backup.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/block/backup.c b/block/backup.c > index

Re: [Qemu-block] [PATCH v9 02/13] block/backup: split shareable copying part from backup_do_cow

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Split copying logic which will be shared with backup-top filter. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/backup.c | 47 --- > 1 file changed, 32 insertions(+), 15

Re: [Qemu-block] [PATCH v5 4/6] iotests: Valgrind fails with nonexistent directory

2019-08-28 Thread Andrey Shinkevich
On 27/08/2019 22:45, John Snow wrote: > > > On 8/25/19 11:24 AM, Andrey Shinkevich wrote: >> >> >> On 16/08/2019 03:55, John Snow wrote: >>> >>> >>> On 7/19/19 12:30 PM, Andrey Shinkevich wrote: The Valgrind uses the exported variable TMPDIR and fails if the directory does not exist.

Re: [Qemu-block] [PATCH v9 02/13] block/backup: split shareable copying part from backup_do_cow

2019-08-28 Thread Vladimir Sementsov-Ogievskiy
28.08.2019 17:22, Max Reitz wrote: > On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: >> Split copying logic which will be shared with backup-top filter. >> >> Signed-off-by: Vladimir Sementsov-Ogievskiy >> --- >> block/backup.c | 47 --- >> 1

Re: [Qemu-block] [PATCH v9 07/13] iotests: prepare 124 and 257 bitmap querying for backup-top filter

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > After backup-top filter appearing it's not possible to see dirty > bitmaps in top node, so use node-name instead. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/124| 3 +- > tests/qemu-iotests/257

Re: [Qemu-block] [PATCH v9 08/13] iotests: 257: drop unused Drive.device field

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > After previous commit Drive.device is actually unused. Drop it together > with .name property. While being here reuse .node in qmp commands > instead of writing 'drive0' twice. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- >

Re: [Qemu-block] [PATCH v9 12/13] block: introduce backup-top filter driver

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Backup-top filter caches write operations and does copy-before-write > operations. > > The driver will be used in backup instead of write-notifiers. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/backup-top.h | 37

Re: [Qemu-block] [PATCH 2/2] qapi: deprecate implicit filters

2019-08-28 Thread John Snow
(Peter: search for "pkrempa" down below.) On 8/28/19 5:20 AM, Vladimir Sementsov-Ogievskiy wrote: > 27.08.2019 23:12, John Snow wrote: >> >> >> On 8/23/19 5:22 AM, Vladimir Sementsov-Ogievskiy wrote: >>> 14.08.2019 13:07, Vladimir Sementsov-Ogievskiy wrote: To get rid of implicit filters

Re: [Qemu-block] [PATCH v9 11/13] block: add lock/unlock range functions

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > From: Vladimir Sementsov-Ogievskiy Hm. :-) Do you want to fix that? > Introduce lock/unlock range functionality, based on serialized > requests. This is needed to refactor backup, dropping local > tracked-request-like synchronization. >

Re: [Qemu-block] [PATCH v5 6/6] iotests: extend sleeping time under Valgrind

2019-08-28 Thread John Snow
On 8/28/19 11:24 AM, Andrey Shinkevich wrote: > > > On 27/08/2019 22:42, John Snow wrote: >> >> >> On 8/23/19 11:27 AM, Vladimir Sementsov-Ogievskiy wrote: >>> 16.08.2019 4:01, John Snow wrote: On 7/19/19 12:30 PM, Andrey Shinkevich wrote: > To synchronize the time when

Re: [Qemu-block] [PATCH v9 09/13] iotests: 257: drop device_add

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > SCSI devices are unused in test, drop them. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > tests/qemu-iotests/257 | 8 --- > tests/qemu-iotests/257.out | 44 -- > 2 files changed, 52

Re: [Qemu-block] [Qemu-devel] [PATCH v6 2/6] iotests: exclude killed processes from running under Valgrind

2019-08-28 Thread John Snow
On 8/26/19 11:50 AM, Andrey Shinkevich wrote: > The Valgrind tool fails to manage its termination in multi-threaded > processes when they raise the signal SIGKILL. The bug has been reported > to the Valgrind maintainers and was registered as the bug #409141: >

Re: [Qemu-block] [Qemu-devel] [PATCH v6 6/6] iotests: extend sleeping time under Valgrind

2019-08-28 Thread John Snow
On 8/26/19 11:50 AM, Andrey Shinkevich wrote: > To synchronize the time when QEMU is running longer under the Valgrind, > increase the sleeping time in the test 247. > > Signed-off-by: Andrey Shinkevich > Reviewed-by: Vladimir Sementsov-Ogievskiy Probably fine, as discussed. We can work on

Re: [Qemu-block] [Qemu-devel] [PATCH v6 1/6] iotests: allow Valgrind checking all QEMU processes

2019-08-28 Thread John Snow
On 8/26/19 11:50 AM, Andrey Shinkevich wrote: > With the '-valgrind' option, let all the QEMU processes be run under > the Valgrind tool. The Valgrind own parameters may be set with its > environment variable VALGRIND_OPTS, e.g. > $ VALGRIND_OPTS="--leak-check=yes" ./check -valgrind > or they

Re: [Qemu-block] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

2019-08-28 Thread John Snow
On 8/28/19 5:34 AM, Thomas Huth wrote: > In our documentation, we use a mix of "$QEMU", "qemu-system-i386" and > "qemu-system-x86_64" when we give examples to the users how to run > QEMU. Some more consistency would be good here. Also some distributions > use different names for the QEMU binary

Re: [Qemu-block] [PATCH v2 0/2] Alignment checks cleanup

2019-08-28 Thread John Snow
On 8/27/19 2:59 PM, Nir Soffer wrote: > While working on 4k support, I noticed that there is lot of code using > BDRV_SECTOR_SIZE (512) for checking alignment. I wonder how this can work with > 4k storage. > > Lets start by cleaning up to make the code easier to understand: > - Use

Re: [Qemu-block] [PATCH v2 2/2] block: Remove unused masks

2019-08-28 Thread John Snow
On 8/27/19 2:59 PM, Nir Soffer wrote: > Replace confusing usage: > > ~BDRV_SECTOR_MASK > > With more clear: > > (BDRV_SECTOR_SIZE - 1) > > Remove BDRV_SECTOR_MASK and the unused BDRV_BLOCK_OFFSET_MASK which was > it's last user. > Kind of lateral in my opinion, but if there was

Re: [Qemu-block] [PATCH 2/2] iotests: Test reverse sub-cluster qcow2 writes

2019-08-28 Thread John Snow
On 8/23/19 9:03 AM, Max Reitz wrote: > This exercises the regression introduced in commit > 50ba5b2d994853b38fed10e0841b119da0f8b8e5. On my machine, it has close > to a 50 % false-negative rate, but that should still be sufficient to > test the fix. > > Signed-off-by: Max Reitz > --- >

Re: [Qemu-block] [PATCH 1/2] block/file-posix: Reduce xfsctl() use

2019-08-28 Thread John Snow
On 8/23/19 9:03 AM, Max Reitz wrote: > This patch removes xfs_write_zeroes() and xfs_discard(). Both functions > have been added just before the same feature was present through > fallocate(): > > - fallocate() has supported PUNCH_HOLE for XFS since Linux 2.6.38 (March > 2011);

Re: [Qemu-block] [PATCH v9 13/13] block/backup: use backup-top instead of write notifiers

2019-08-28 Thread Max Reitz
On 26.08.19 18:13, Vladimir Sementsov-Ogievskiy wrote: > Drop write notifiers and use filter node instead. > > = Changes = > > 1. add filter-node-name argument for backup qmp api. We have to do it > in this commit, as 257 needs to be fixed. I feel a bit bad about it not being an implicit node.

Re: [Qemu-block] [Qemu-devel] [PATCH] qemu-doc: Do not hard-code the name of the QEMU binary

2019-08-28 Thread Thomas Huth
On 28/08/2019 21.18, John Snow wrote: > > > On 8/28/19 5:34 AM, Thomas Huth wrote: >> In our documentation, we use a mix of "$QEMU", "qemu-system-i386" and >> "qemu-system-x86_64" when we give examples to the users how to run >> QEMU. Some more consistency would be good here. Also some

Re: [Qemu-block] [Qemu-devel] [PATCH v6 1/6] iotests: allow Valgrind checking all QEMU processes

2019-08-28 Thread Eric Blake
On 8/28/19 5:58 PM, John Snow wrote: >> +++ b/tests/qemu-iotests/common.rc >> @@ -60,61 +60,132 @@ if ! . ./common.config >> exit 1 >> fi >> >> +# Unset the variables to turn Valgrind off for specific processes, e.g. That's not unsetting, that's setting to the empty string. >> +# $

Re: [Qemu-block] [Qemu-devel] [PATCH v2] job: drop job_drain

2019-08-28 Thread John Snow
On 8/23/19 10:32 AM, Vladimir Sementsov-Ogievskiy wrote: > In job_finish_sync job_enter should be enough for a job to make some > progress and draining is a wrong tool for it. So use job_enter directly > here and drop job_drain with all related staff not used more. > > Suggested-by: Kevin Wolf

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/4] iotests: use python logging

2019-08-28 Thread John Snow
Gentle ping. This should be fairly easy to review, I hope; the worst of it is making sure that no tests remain that don't engage an entry point in iotests.py anymore. On 8/20/19 7:52 PM, John Snow wrote: > This series uses python logging to enable output conditionally on > iotests.log(). We unify