Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-03 Thread John Snow
I have clearly been lax in checking my coding style... I'll fix these, but not until review. On 7/3/19 9:50 PM, no-re...@patchew.org wrote: > Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/ > > > > Hi, > > This series seems to have some coding style problems.

Re: [Qemu-block] [Qemu-devel] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-03 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190703224707.12437-1-ebl...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash make

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-03 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash make

Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-03 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Type: series Subject: [Qemu-devel] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode Message-id:

[Qemu-block] [PATCH v2 RFC] qemu-nbd: Permit TLS with Unix sockets

2019-07-03 Thread Eric Blake
Although you generally won't use encryption with a Unix socket (after all, everything is local, so why waste the CPU power), there are situations in testsuites where Unix sockets are much nicer than TCP sockets. Since nbdkit allows encryption over both types of sockets, it makes sense for

[Qemu-block] [PATCH v2 14/18] iotests: teach run_job to cancel pending jobs

2019-07-03 Thread John Snow
run_job can cancel pending jobs to simulate failure. This lets us use the pending callback to issue test commands while the job is open, but then still have the job fail in the end. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 22 -- 1 file changed, 20

[Qemu-block] [PATCH v2 10/18] block/dirty-bitmap: add bdrv_dirty_bitmap_get

2019-07-03 Thread John Snow
Add a public interface for get. While we're at it, rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked". (There are more functions to rename to the bdrv_dirty_bitmap_VERB form, but they will wait until the conclusion of this series.) Signed-off-by: John Snow ---

[Qemu-block] [PATCH v2 08/18] hbitmap: enable merging across granularities

2019-07-03 Thread John Snow
Signed-off-by: John Snow --- util/hbitmap.c | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git a/util/hbitmap.c b/util/hbitmap.c index 3b6acae42b..306bc4876d 100644 --- a/util/hbitmap.c +++ b/util/hbitmap.c @@ -777,7 +777,27 @@ void

[Qemu-block] [PATCH v2 18/18] block/backup: loosen restriction on readonly bitmaps

2019-07-03 Thread John Snow
With the "never" sync policy, we actually can utilize readonly bitmaps now. Loosen the check at the QMP level, and tighten it based on provided arguments down at the job creation level instead. Signed-off-by: John Snow --- block/backup.c | 6 ++ blockdev.c | 2 +- 2 files changed, 7

[Qemu-block] [PATCH v2 07/18] hbitmap: Fix merge when b is empty, and result is not an alias of a

2019-07-03 Thread John Snow
Nobody calls the function like this currently, but we neither prohibit or cope with this behavior. I decided to make the function cope with it. Signed-off-by: John Snow --- util/hbitmap.c | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/util/hbitmap.c

[Qemu-block] [PATCH v2 05/18] block/backup: Add mirror sync mode 'bitmap'

2019-07-03 Thread John Snow
We don't need or want a new sync mode for simple differences in semantics. Create a new mode simply named "BITMAP" that is designed to make use of the new Bitmap Sync Mode field. Because the only bitmap mode is 'conditional', this adds no new functionality to the backup job (yet). The old

[Qemu-block] [PATCH v2 06/18] block/backup: add 'never' policy to bitmap sync mode

2019-07-03 Thread John Snow
This adds a "never" policy for bitmap synchronization. Regardless of if the job succeeds or fails, we never update the bitmap. This can be used to perform differential backups, or simply to avoid the job modifying a bitmap. Signed-off-by: John Snow --- block/backup.c | 5 +++--

[Qemu-block] [PATCH v2 17/18] iotests: add test 257 for bitmap-mode backups

2019-07-03 Thread John Snow
Signed-off-by: John Snow --- tests/qemu-iotests/257 | 409 +++ tests/qemu-iotests/257.out | 2199 tests/qemu-iotests/group |1 + 3 files changed, 2609 insertions(+) create mode 100755 tests/qemu-iotests/257 create mode 100644

[Qemu-block] [PATCH v2 12/18] block/backup: add 'always' bitmap sync policy

2019-07-03 Thread John Snow
This adds an "always" policy for bitmap synchronization. Regardless of if the job succeeds or fails, the bitmap is *always* synchronized. This means that for backups that fail part-way through, the bitmap retains a record of which sectors need to be copied out to accomplish a new backup using the

[Qemu-block] [PATCH v2 13/18] iotests: add testing shim for script-style python tests

2019-07-03 Thread John Snow
Because the new-style python tests don't use the iotests.main() test launcher, we don't turn on the debugger logging for these scripts when invoked via ./check -d. Refactor the launcher shim into new and old style shims so that they share environmental configuration. Two cleanup notes: debug was

[Qemu-block] [PATCH v2 04/18] qapi: add BitmapSyncMode enum

2019-07-03 Thread John Snow
Depending on what a user is trying to accomplish, there might be a few bitmap cleanup actions that occur when an operation is finished that could be useful. I am proposing three: - NEVER: The bitmap is never synchronized against what was copied. - ALWAYS: The bitmap is always synchronized, even

[Qemu-block] [PATCH v2 16/18] iotests: Add virtio-scsi device helper

2019-07-03 Thread John Snow
Seems that it comes up enough. Signed-off-by: John Snow --- tests/qemu-iotests/040| 6 +- tests/qemu-iotests/093| 6 ++ tests/qemu-iotests/139| 7 ++- tests/qemu-iotests/238| 5 + tests/qemu-iotests/iotests.py | 4 5 files changed, 10

[Qemu-block] [PATCH v2 01/18] qapi/block-core: Introduce BackupCommon

2019-07-03 Thread John Snow
drive-backup and blockdev-backup have an awful lot of things in common that are the same. Let's fix that. I don't deduplicate 'target', because the semantics actually did change between each structure. Leave that one alone so it can be documented separately. Signed-off-by: John Snow ---

[Qemu-block] [PATCH v2 03/18] blockdev-backup: utilize do_backup_common

2019-07-03 Thread John Snow
Signed-off-by: John Snow --- blockdev.c | 73 +++--- 1 file changed, 3 insertions(+), 70 deletions(-) diff --git a/blockdev.c b/blockdev.c index 5fd663a7e5..cad300d526 100644 --- a/blockdev.c +++ b/blockdev.c @@ -3638,82 +3638,15 @@ XDbgBlockGraph

[Qemu-block] [PATCH v2 09/18] block/dirty-bitmap: add bdrv_dirty_bitmap_merge_internal

2019-07-03 Thread John Snow
I'm surprised it didn't come up sooner, but sometimes we have a +busy bitmap as a source. This is dangerous from the QMP API, but if we are the owner that marked the bitmap busy, it's safe to merge it using it as a read only source. It is not safe in the general case to allow users to read from

[Qemu-block] [PATCH v2 11/18] block/backup: upgrade copy_bitmap to BdrvDirtyBitmap

2019-07-03 Thread John Snow
This simplifies some interface matters; namely the initialization and (later) merging the manifest back into the sync_bitmap if it was provided. Signed-off-by: John Snow --- block/backup.c | 76 -- 1 file changed, 37 insertions(+), 39 deletions(-)

[Qemu-block] [PATCH v2 15/18] iotests: teach FilePath to produce multiple paths

2019-07-03 Thread John Snow
Use "FilePaths" instead of "FilePath" to request multiple files be cleaned up after we leave that object's scope. This is not crucial; but it saves a little typing. Signed-off-by: John Snow --- tests/qemu-iotests/iotests.py | 33 ++--- 1 file changed, 22

[Qemu-block] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode

2019-07-03 Thread John Snow
This series adds a new "BITMAP" sync mode that is meant to replace the existing "INCREMENTAL" sync mode. This mode can have its behavior modified by issuing any of three bitmap sync modes, passed as arguments to the job. The three bitmap sync modes are: - CONDITIONAL: This is an alias for the

[Qemu-block] [PATCH v2 02/18] drive-backup: create do_backup_common

2019-07-03 Thread John Snow
Create a common core that comprises the actual meat of what the backup API boundary needs to do, and then switch drive-backup to use it. Questions: - do_drive_backup now acquires and releases the aio_context in addition to do_backup_common doing the same. Can I drop this from drive_backup?

Re: [Qemu-block] [Qemu-devel] [PATCH v3 0/6] Few fixes for userspace NVME driver

2019-07-03 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20190703155944.9637-1-mlevi...@redhat.com/ Hi, This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN === #!/bin/bash make

Re: [Qemu-block] [PATCH v2 2/3] qapi: implement block-dirty-bitmap-remove transaction action

2019-07-03 Thread John Snow
On 7/3/19 3:30 PM, Max Reitz wrote: > On 01.07.19 22:13, John Snow wrote: >> It is used to do transactional movement of the bitmap (which is >> possible in conjunction with merge command). Transactional bitmap >> movement is needed in scenarios with external snapshot, when we don't >> want to

Re: [Qemu-block] [PATCH v2 2/3] qapi: implement block-dirty-bitmap-remove transaction action

2019-07-03 Thread Max Reitz
On 03.07.19 21:30, Max Reitz wrote: > On 01.07.19 22:13, John Snow wrote: >> It is used to do transactional movement of the bitmap (which is >> possible in conjunction with merge command). Transactional bitmap >> movement is needed in scenarios with external snapshot, when we don't >> want to

Re: [Qemu-block] [PATCH v2 2/3] qapi: implement block-dirty-bitmap-remove transaction action

2019-07-03 Thread Max Reitz
On 01.07.19 22:13, John Snow wrote: > It is used to do transactional movement of the bitmap (which is > possible in conjunction with merge command). Transactional bitmap > movement is needed in scenarios with external snapshot, when we don't > want to leave copy of the bitmap in the base image. >

[Qemu-block] [PATCH v2 08/12] iotests: Compare error messages in 030

2019-07-03 Thread Max Reitz
Currently, 030 just compares the error class, which does not say anything. Before HEAD^ added throttling to test_overlapping_4, that test actually usually failed because node2 was already gone, not because it was the commit and stream job were not allowed to overlap. Prevent such problems in the

[Qemu-block] [PATCH v2 07/12] iotests: Fix throttling in 030

2019-07-03 Thread Max Reitz
Currently, TestParallelOps in 030 creates images that are too small for job throttling to be effective. This is reflected by the fact that it never undoes the throttling. Increase the image size and undo the throttling when the job should be completed. Also, add throttling in

[Qemu-block] [PATCH v2 03/12] block/stream: Swap backing file change order

2019-07-03 Thread Max Reitz
bdrv_change_backing_file() can result in yields. Therefore, @base may no longer be the the backing_bs() of s->bottom afterwards. Just swap the order of the two calls to fix this. Signed-off-by: Max Reitz --- block/stream.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git

Re: [Qemu-block] [PATCH v3 7/7] iotests: Set read-zeroes on in null block driver for Valgrind

2019-07-03 Thread Andrey Shinkevich
On 28/06/2019 16:46, Andrey Shinkevich wrote: > The Valgrind tool reports about the uninitialised buffer 'buf' > instantiated on the stack of the function guess_disk_lchs(). > It is revealed in the tests 051, 186, 227 and 240. > Pass 'read-zeroes=on' to the null block driver in the mentioned

[Qemu-block] [PATCH v2 10/12] iotests: Add new case to 030

2019-07-03 Thread Max Reitz
We recently removed the dependency of the stream job on its base node. That makes it OK to use a commit filter node there. Test that. Signed-off-by: Max Reitz Tested-by: Andrey Shinkevich Reviewed-by: Alberto Garcia --- tests/qemu-iotests/030 | 25 +

[Qemu-block] [PATCH v2 06/12] block: Deep-clear inherits_from

2019-07-03 Thread Max Reitz
BDS.inherits_from does not always point to an immediate parent node. When launching a block job with a filter node, for example, the node directly below the filter will not point to the filter, but keep its old pointee (above the filter). If that pointee goes away while the job is still running,

[Qemu-block] [PATCH v2 09/12] iotests: Add @use_log to VM.run_job()

2019-07-03 Thread Max Reitz
unittest-style tests generally do not use the log file, but VM.run_job() can still be useful to them. Add a parameter to it that hides its output from the log file. Signed-off-by: Max Reitz --- tests/qemu-iotests/iotests.py | 18 +- 1 file changed, 13 insertions(+), 5

[Qemu-block] [PATCH v2 00/12] block: Fixes for concurrent block jobs

2019-07-03 Thread Max Reitz
This is a v2 to “block: Add BDS.never_freeze”. It depends on my “block: Delay poll when ending drained sections” series: Depends-on: <20190619152603.5937-1-mre...@redhat.com> It turned out that if you run 030 (or just the new test_overlapping_5 case) sufficiently often, it breaks; which is why

[Qemu-block] [PATCH v2 05/12] block: Reduce (un)drains when replacing a child

2019-07-03 Thread Max Reitz
Currently, bdrv_replace_child_noperm() undrains the parent until it is completely undrained, then re-drains it after attaching the new child node. This is a problem with bdrv_drop_intermediate(): We want to keep the whole subtree drained, including parents, while the operation is under way.

[Qemu-block] [PATCH v2 04/12] block: Keep subtree drained in drop_intermediate

2019-07-03 Thread Max Reitz
bdrv_drop_intermediate() calls BdrvChildRole.update_filename(). That may poll, thus changing the graph, which potentially breaks the QLIST_FOREACH_SAFE() loop. Just keep the whole subtree drained. This is probably the right thing to do anyway (dropping nodes while the subtree is not drained

[Qemu-block] [PATCH v2 12/12] iotests: Add test for concurrent stream/commit

2019-07-03 Thread Max Reitz
We already have 030 for that in general, but this tests very specific cases of both jobs finishing concurrently. Signed-off-by: Max Reitz --- tests/qemu-iotests/258 | 163 + tests/qemu-iotests/258.out | 33 tests/qemu-iotests/group | 1 + 3

[Qemu-block] [PATCH v2 11/12] iotests: Add read-only test case to 030

2019-07-03 Thread Max Reitz
This tests that the stream job exits cleanly (without abort) when the top node is read-only and cannot be reopened read/write. Signed-off-by: Max Reitz --- tests/qemu-iotests/030 | 29 - tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 30 insertions(+), 3

[Qemu-block] [PATCH v2 02/12] block/stream: Fix error path

2019-07-03 Thread Max Reitz
As of commit c624b015bf14fe01f1e6452a36e63b3ea1ae4998, the stream job only freezes the chain until the overlay of the base node. The error path must consider this. Fixes: c624b015bf14fe01f1e6452a36e63b3ea1ae4998 Signed-off-by: Max Reitz --- block/stream.c | 2 +- 1 file changed, 1

[Qemu-block] [PATCH v2 01/12] block: Add BDS.never_freeze

2019-07-03 Thread Max Reitz
The commit and the mirror block job must be able to drop their filter node at any point. However, this will not be possible if any of the BdrvChild links to them is frozen. Therefore, we need to prevent them from ever becoming frozen. Signed-off-by: Max Reitz Reviewed-by: Andrey Shinkevich

Re: [Qemu-block] [PULL 27/27] hw/block/pflash_cfi02: Reduce I/O accesses to 16-bit

2019-07-03 Thread Stephen Checkoway
> On Jul 3, 2019, at 12:02, Philippe Mathieu-Daudé wrote: > > On 7/3/19 5:52 PM, Stephen Checkoway wrote: >> >> >>> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote: >>> >>> Parallel NOR flashes are limited to 16-bit bus accesses. >> >> I don't think this is correct. The CFI spec

Re: [Qemu-block] [PULL 27/27] hw/block/pflash_cfi02: Reduce I/O accesses to 16-bit

2019-07-03 Thread Philippe Mathieu-Daudé
On 7/3/19 5:52 PM, Stephen Checkoway wrote: > > >> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote: >> >> Parallel NOR flashes are limited to 16-bit bus accesses. > > I don't think this is correct. The CFI spec defines an x32 interface for > parallel NOR. CFI addresses 0x28 and 0x29

Re: [Qemu-block] [PULL 27/27] hw/block/pflash_cfi02: Reduce I/O accesses to 16-bit

2019-07-03 Thread Philippe Mathieu-Daudé
On 7/3/19 6:20 PM, Stephen Checkoway wrote: >> On Jul 3, 2019, at 12:02, Philippe Mathieu-Daudé wrote: >> On 7/3/19 5:52 PM, Stephen Checkoway wrote: >>> >>> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote: Parallel NOR flashes are limited to 16-bit bus accesses. >>> >>> I

[Qemu-block] [PATCH v4] block/nvme: add support for discard

2019-07-03 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/nvme.c | 81 ++ block/trace-events | 2 ++ 2 files changed, 83 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index 02e0846643..96a715dcc1 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -111,6

Re: [Qemu-block] [PATCH v1 3/3] qcow2: add zstd cluster compression

2019-07-03 Thread Kevin Wolf
Am 03.07.2019 um 16:36 hat Eric Blake geschrieben: > On 7/3/19 6:00 AM, Denis Plotnikov wrote: > > zstd significantly reduces cluster compression time. > > It provides better compression performance maintaining > > the same level of compression ratio in comparison with > > zlib, which, by the

[Qemu-block] [PATCH v3 3/6] block/nvme: support larger that 512 bytes sector devices

2019-07-03 Thread Maxim Levitsky
Currently the driver hardcodes the sector size to 512, and doesn't check the underlying device. Fix that. Also fail if underlying nvme device is formatted with metadata as this needs special support. Signed-off-by: Maxim Levitsky --- block/nvme.c | 45

[Qemu-block] [PATCH v3 6/6] block/nvme: add support for discard

2019-07-03 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/nvme.c | 81 ++ block/trace-events | 2 ++ 2 files changed, 83 insertions(+) diff --git a/block/nvme.c b/block/nvme.c index 02e0846643..f8bcf1ffb6 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -111,6

[Qemu-block] [PATCH v3 0/6] Few fixes for userspace NVME driver

2019-07-03 Thread Maxim Levitsky
Compared to last submission, this series adds another patch, which implements support for image creation over the nvme drive like that: qemu-img create -f qcow2 nvme://:06:00.0/1 10G -o preallocation=metadata I also addressed the review comments. Best regards, Maxim Levitsky Maxim

[Qemu-block] [PATCH v3 4/6] block/nvme: add support for image creation

2019-07-03 Thread Maxim Levitsky
Tesed on a nvme device like that: # create preallocated qcow2 image $ qemu-img create -f qcow2 nvme://:06:00.0/1 10G -o preallocation=metadata Formatting 'nvme://:06:00.0/1', fmt=qcow2 size=10737418240 cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16 #

[Qemu-block] [PATCH v3 5/6] block/nvme: add support for write zeros

2019-07-03 Thread Maxim Levitsky
Signed-off-by: Maxim Levitsky --- block/nvme.c | 69 +++- block/trace-events | 1 + include/block/nvme.h | 19 +++- 3 files changed, 87 insertions(+), 2 deletions(-) diff --git a/block/nvme.c b/block/nvme.c index 152d27b07f..02e0846643

[Qemu-block] [PATCH v3 2/6] block/nvme: fix doorbell stride

2019-07-03 Thread Maxim Levitsky
Fix the math involving non standard doorbell stride Signed-off-by: Maxim Levitsky --- block/nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/nvme.c b/block/nvme.c index 6d4e7f3d83..52798081b2 100644 --- a/block/nvme.c +++ b/block/nvme.c @@ -217,7 +217,7 @@ static

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Denis Plotnikov
On 03.07.2019 18:46, Kevin Wolf wrote: > Am 03.07.2019 um 17:01 hat Denis Plotnikov geschrieben: >> On 03.07.2019 17:14, Eric Blake wrote: >>> On 7/3/19 6:00 AM, Denis Plotnikov wrote: diff --git a/block/qcow2.c b/block/qcow2.c index 3ace3b2209..921eb67b80 100644 ---

[Qemu-block] [PATCH v3 1/6] block/nvme: don't touch the completion entries

2019-07-03 Thread Maxim Levitsky
Completion entries are meant to be only read by the host and written by the device. The driver is supposed to scan the completions from the last point where it left, and until it sees a completion with non flipped phase bit. Signed-off-by: Maxim Levitsky --- block/nvme.c | 5 + 1 file

Re: [Qemu-block] [PULL 27/27] hw/block/pflash_cfi02: Reduce I/O accesses to 16-bit

2019-07-03 Thread Stephen Checkoway
> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote: > > Parallel NOR flashes are limited to 16-bit bus accesses. I don't think this is correct. The CFI spec defines an x32 interface for parallel NOR. CFI addresses 0x28 and 0x29 specify the interface and value 3 is x32 and value 5 is

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Kevin Wolf
Am 03.07.2019 um 17:01 hat Denis Plotnikov geschrieben: > On 03.07.2019 17:14, Eric Blake wrote: > > On 7/3/19 6:00 AM, Denis Plotnikov wrote: > >> diff --git a/block/qcow2.c b/block/qcow2.c > >> index 3ace3b2209..921eb67b80 100644 > >> --- a/block/qcow2.c > >> +++ b/block/qcow2.c > >> @@ -1202,6

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Denis Plotnikov
On 03.07.2019 18:13, Eric Blake wrote: > On 7/3/19 10:01 AM, Denis Plotnikov wrote: > + * with QCOW2_COMPRESSION_TYPE_ZLIB the corresponding incompatible + * feature flag must be absent, with other compression types the + * incompatible feature flag must be set >>> Is

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] raw-posix.c - use max transfer length / max segemnt count only for SCSI passthrough

2019-07-03 Thread Maxim Levitsky
On Wed, 2019-07-03 at 09:50 -0500, Eric Blake wrote: > On 6/30/19 10:08 AM, Maxim Levitsky wrote: > > Regular block devices (/dev/sda*, /dev/nvme*, etc) interface is not limited > > The regular block device interface is > > or > > Regular block devices interfaces are > > > by the underlying

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Eric Blake
On 7/3/19 10:01 AM, Denis Plotnikov wrote: >>> + * with QCOW2_COMPRESSION_TYPE_ZLIB the corresponding incompatible >>> + * feature flag must be absent, with other compression types the >>> + * incompatible feature flag must be set >> Is there a strong reason for forbid the

Re: [Qemu-block] [PATCH v1 3/3] qcow2: add zstd cluster compression

2019-07-03 Thread Denis Plotnikov
On 03.07.2019 17:36, Eric Blake wrote: > On 7/3/19 6:00 AM, Denis Plotnikov wrote: >> zstd significantly reduces cluster compression time. >> It provides better compression performance maintaining >> the same level of compression ratio in comparison with >> zlib, which, by the moment, has been

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Denis Plotnikov
On 03.07.2019 17:14, Eric Blake wrote: > On 7/3/19 6:00 AM, Denis Plotnikov wrote: >> The patch adds some preparation parts for incompatible compression type >> feature to QCOW2 header that indicates that*all* compressed clusters >> must be (de)compressed using a certain compression type. >> >>

Re: [Qemu-block] [Qemu-devel] [PATCH 1/1] raw-posix.c - use max transfer length / max segemnt count only for SCSI passthrough

2019-07-03 Thread Eric Blake
On 6/30/19 10:08 AM, Maxim Levitsky wrote: > Regular block devices (/dev/sda*, /dev/nvme*, etc) interface is not limited The regular block device interface is or Regular block devices interfaces are > by the underlying storage limits, but rather the kernel block layer > takes care to split the

Re: [Qemu-block] [PATCH 0/1] RFC: don't obey the block device max transfer len / max segments for block devices

2019-07-03 Thread Eric Blake
On 7/3/19 4:52 AM, Stefan Hajnoczi wrote: > On Sun, Jun 30, 2019 at 06:08:54PM +0300, Maxim Levitsky wrote: >> It looks like Linux block devices, even in O_DIRECT mode don't have any user >> visible >> limit on transfer size / number of segments, which underlying block device >> can have. >> The

Re: [Qemu-block] [PATCH v1 3/3] qcow2: add zstd cluster compression

2019-07-03 Thread Eric Blake
On 7/3/19 6:00 AM, Denis Plotnikov wrote: > zstd significantly reduces cluster compression time. > It provides better compression performance maintaining > the same level of compression ratio in comparison with > zlib, which, by the moment, has been the only compression > method available. > >

Re: [Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Eric Blake
On 7/3/19 6:00 AM, Denis Plotnikov wrote: > The patch adds some preparation parts for incompatible compression type > feature to QCOW2 header that indicates that *all* compressed clusters > must be (de)compressed using a certain compression type. > > It is implied that the compression type is set

[Qemu-block] [PATCH v1 0/3] add zstd cluster compression

2019-07-03 Thread Denis Plotnikov
change log: v1: * extend qcow2 header instead of adding a new incompatible extension header specification re-written accordingly * enable zstd compression via config * fix zstd (de)compression functions * fix comments/description * fix function naming --- The goal of

[Qemu-block] [PATCH v1 3/3] qcow2: add zstd cluster compression

2019-07-03 Thread Denis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of compression ratio in comparison with zlib, which, by the moment, has been the only compression method available. The performance test results: Test compresses and

[Qemu-block] [PATCH v1 1/3] qcow2: introduce compression type feature

2019-07-03 Thread Denis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to QCOW2 header that indicates that *all* compressed clusters must be (de)compressed using a certain compression type. It is implied that the compression type is set on the image creation and can be changed only later

[Qemu-block] [PATCH v1 2/3] qcow2: rework the cluster compression routine

2019-07-03 Thread Denis Plotnikov
The patch allow to process image compression type defined in the image header and choose an appropriate method for image clusters (de)compression. Signed-off-by: Denis Plotnikov --- block/qcow2.c | 93 --- 1 file changed, 73 insertions(+), 20

Re: [Qemu-block] [PATCH 0/1] RFC: don't obey the block device max transfer len / max segments for block devices

2019-07-03 Thread Stefan Hajnoczi
On Sun, Jun 30, 2019 at 06:08:54PM +0300, Maxim Levitsky wrote: > It looks like Linux block devices, even in O_DIRECT mode don't have any user > visible > limit on transfer size / number of segments, which underlying block device > can have. > The block layer takes care of enforcing these limits

Re: [Qemu-block] [PATCH v2] block/rbd: implement .bdrv_get_allocated_file_size callback

2019-07-03 Thread Stefano Garzarella
On Tue, Jul 02, 2019 at 11:09:14AM -0400, Jason Dillaman wrote: > On Fri, Jun 28, 2019 at 4:59 AM Stefano Garzarella > wrote: > > > > On Thu, Jun 27, 2019 at 03:43:04PM -0400, Jason Dillaman wrote: > > > On Thu, Jun 27, 2019 at 1:24 PM John Snow wrote: > > > > On 6/27/19 4:48 AM, Stefano

Re: [Qemu-block] [Qemu-devel] [PATCH v5 02/12] qapi/block-core: add option for io_uring

2019-07-03 Thread Markus Armbruster
I apologize for replying so late. Stefan Hajnoczi writes: > On Tue, Jun 11, 2019 at 03:36:53PM +0800, Fam Zheng wrote: >> On Mon, 06/10 19:18, Aarushi Mehta wrote: >> > Option only enumerates for hosts that support it. >> > >> > Signed-off-by: Aarushi Mehta >> > Reviewed-by: Stefan Hajnoczi