Re: [PATCH] doc: Describe missing generic -blockdev options

2019-10-15 Thread Peter Maydell
On Tue, 15 Oct 2019 at 13:40, Kevin Wolf wrote: > > We added more generic options after introducing -blockdev and forgot to > update the documentation (man page and --help output) accordingly. Do > that now. > > Signed-off-by: Kevin Wolf > --- > qemu-options.hx | 19 ++- > 1

Re: [PATCH] doc: Describe missing generic -blockdev options

2019-10-15 Thread Eric Blake
On 10/15/19 7:38 AM, Kevin Wolf wrote: We added more generic options after introducing -blockdev and forgot to update the documentation (man page and --help output) accordingly. Do that now. Signed-off-by: Kevin Wolf --- qemu-options.hx | 19 ++- 1 file changed, 18

Re: [PATCH] blockdev: Use error_report() in hmp_commit()

2019-10-15 Thread Eric Blake
On 10/15/19 7:39 AM, Kevin Wolf wrote: Instead of using monitor_printf() to report errors, hmp_commit() should use error_report() like other places do. Signed-off-by: Kevin Wolf --- blockdev.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) Reviewed-by: Eric Blake -- Eric

Re: [PATCH] doc: Describe missing generic -blockdev options

2019-10-15 Thread Kevin Wolf
Am 15.10.2019 um 15:55 hat Peter Maydell geschrieben: > On Tue, 15 Oct 2019 at 13:40, Kevin Wolf wrote: > > > > We added more generic options after introducing -blockdev and forgot to > > update the documentation (man page and --help output) accordingly. Do > > that now. > > > > Signed-off-by:

Re: [PATCH] doc: Describe missing generic -blockdev options

2019-10-15 Thread Eric Blake
On 10/15/19 9:05 AM, Kevin Wolf wrote: +@item force-share +Override the image locking system of QEMU and force the node to allowing +sharing all permissions with other uses. Grammar nit: "to allow sharing"; but maybe the phrasing could be clarified anyway -- I'm not entirely sure what

[PATCH v2 02/21] iotests/qcow2.py: Split feature fields into bits

2019-10-15 Thread Max Reitz
Print the feature fields as a set of bits so that filtering is easier. Signed-off-by: Max Reitz --- tests/qemu-iotests/031.out | 36 +-- tests/qemu-iotests/036.out | 18 +- tests/qemu-iotests/039.out | 22 ++-- tests/qemu-iotests/060.out | 20 +--

[PATCH v2 05/21] iotests: Replace IMGOPTS by _unsupported_imgopts

2019-10-15 Thread Max Reitz
Some tests require compat=1.1 and thus set IMGOPTS='compat=1.1' globally. That is not how it should be done; instead, they should simply set _unsupported_imgopts to compat=0.10 (compat=1.1 is the default anyway). This makes the tests heed user-specified $IMGOPTS. Some do not work with all image

[PATCH v2 01/21] iotests/qcow2.py: Add dump-header-exts

2019-10-15 Thread Max Reitz
This is useful for tests that want to whitelist fields from dump-header (with grep) but still print all header extensions. Signed-off-by: Max Reitz --- tests/qemu-iotests/qcow2.py | 5 + 1 file changed, 5 insertions(+) diff --git a/tests/qemu-iotests/qcow2.py b/tests/qemu-iotests/qcow2.py

[PATCH v2 00/21] iotests: Allow ./check -o data_file

2019-10-15 Thread Max Reitz
Hi, The cover letter from v1 (explaining the motivation behind this series and the general structure) is here: https://lists.nongnu.org/archive/html/qemu-block/2019-09/msg01323.html For v2, I’ve tried to address Maxim’s comments: - Patch 1 through 3: New - Patch 4: Only print feature bits

[PATCH v2 03/21] iotests: Add _filter_json_filename

2019-10-15 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/common.filter | 24 1 file changed, 24 insertions(+) diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter index 9f418b4881..63bc6f6f26 100644 --- a/tests/qemu-iotests/common.filter +++

[PATCH v3 1/5] qcow2: Allow writing compressed data of multiple clusters

2019-10-15 Thread Andrey Shinkevich
QEMU currently supports writing compressed data of the size equal to one cluster. This patch allows writing QCOW2 compressed data that exceed one cluster. Now, we split buffered data into separate clusters and write them compressed using the existing functionality. To inform the block layer about

[PATCH v3 2/5] tests/qemu-iotests: add case to write compressed data of multiple clusters

2019-10-15 Thread Andrey Shinkevich
Add the test case to the iotest #214 that checks possibility of writing compressed data of more than one cluster size. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/214 | 35 +++ tests/qemu-iotests/214.out | 15 +++ 2 files changed, 50

[PATCH v2 3/3] tests: More iotest 223 improvements

2019-10-15 Thread Eric Blake
Run the core of the test twice, once without iothreads, and again with, for more coverage of both setups. Suggested-by: Nir Soffer Signed-off-by: Eric Blake --- tests/qemu-iotests/223 | 16 ++- tests/qemu-iotests/223.out | 85 +- 2 files changed, 97

[PATCH v2 2/3] iotests: Include QMP input in .out files

2019-10-15 Thread Eric Blake
We generally include relevant HMP input in .out files, by virtue of the fact that HMP echoes its input. But QMP does not, so we have to explicitly inject it in the output stream, in order to make it easier to read .out files to see what behavior is being tested (especially true where the output

[PATCH v3 0/5] qcow2: advanced compression options

2019-10-15 Thread Andrey Shinkevich
New enhancements for writing compressed data to QCOW2 image. The preceding patches have been queued in the Max's block branch: Based-on: <20190916175324.18478-1-vsement...@virtuozzo.com> v2: Instead of introducing multiple key options for many drivers, the 'compression' option has been

[PATCH v3 5/5] tests/qemu-iotests: add case for block-stream compress

2019-10-15 Thread Andrey Shinkevich
Add a case to the iotest #030 that tests the 'compress' option for a block-stream job. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/030 | 51 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 52 insertions(+), 3 deletions(-)

[PATCH v3 4/5] block-stream: add compress option

2019-10-15 Thread Andrey Shinkevich
Allow data compression during block-stream job for backup backing chain. Signed-off-by: Andrey Shinkevich --- block/stream.c | 10 -- blockdev.c | 12 +++- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/block/stream.c b/block/stream.c index 5562ccb..25f9324

[PATCH v3 3/5] block: support compressed write for copy-on-read

2019-10-15 Thread Andrey Shinkevich
Support the data compression during block-stream job over a backup backing chain implemented in the following patch 'block-stream: add compress option'. Signed-off-by: Anton Nefedov Signed-off-by: Denis V. Lunev Signed-off-by: Andrey Shinkevich --- block/io.c | 21

[PATCH v2 0/3] tests: More iotest 223 improvements

2019-10-15 Thread Eric Blake
[subject line kept for continuity with v1, but now touches much more] Max suggested that instead of special-casing just 223 to trace QMP input as well output, that we should instead patch common.qemu to do it for all tests. That in turn found that test 173 has been broken since v3.0. Max also

[PATCH v2 16/21] iotests: Make 091 work with data_file

2019-10-15 Thread Max Reitz
The image end offset as reported by qemu-img check is different when using an external data file; we do not care about its value here, so we can just filter it. Incidentally, common.rc already has _check_test_img for us which does exactly that. Signed-off-by: Max Reitz ---

[PATCH v2 08/21] iotests: Add -o and --no-opts to _make_test_img

2019-10-15 Thread Max Reitz
Blindly overriding IMGOPTS is suboptimal as this discards user-specified options. Whatever options the test needs should simply be appended. Some tests do this (with IMGOPTS=$(_optstr_add "$IMGOPTS" "...")), but that is cumbersome. It’s simpler to just give _make_test_img an -o parameter with

[PATCH v2 11/21] iotests: Replace IMGOPTS='' by --no-opts

2019-10-15 Thread Max Reitz
Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/071 | 4 ++-- tests/qemu-iotests/174 | 2 +- tests/qemu-iotests/178 | 4 ++-- tests/qemu-iotests/197 | 4 ++-- tests/qemu-iotests/215 | 4 ++-- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git

[PATCH v2 15/21] iotests: Avoid cp/mv of test images

2019-10-15 Thread Max Reitz
This will not work with external data files, so try to get tests working without it as far as possible. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/063 | 12 tests/qemu-iotests/063.out | 3 ++- tests/qemu-iotests/085 | 9 +++--

[PATCH v2 09/21] iotests: Inject space into -ocompat=0.10 in 051

2019-10-15 Thread Max Reitz
It did not matter before, but now that _make_test_img understands -o, we should use it properly here. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/051 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qemu-iotests/051

Re: [PATCH] blockdev: Use error_report() in hmp_commit()

2019-10-15 Thread Philippe Mathieu-Daudé
On 10/15/19 2:39 PM, Kevin Wolf wrote: Instead of using monitor_printf() to report errors, hmp_commit() should use error_report() like other places do. Signed-off-by: Kevin Wolf --- blockdev.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/blockdev.c b/blockdev.c

Re: [PATCH v2 00/20] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-10-15 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20191015103900.313928-1-...@irrelevant.dk/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v2 00/20] nvme: support NVMe v1.3d, SGLs and multiple namespaces Type: series Message-id:

Re: [PATCH v2 6/6] tests/qemu-iotests: add case for block-stream compress

2019-10-15 Thread Andrey Shinkevich
On 03/10/2019 17:58, Vladimir Sementsov-Ogievskiy wrote: > 02.10.2019 17:22, Andrey Shinkevich wrote: >> Add a test case to the iotest #030 that checks 'compress' option for a >> block-stream job. >> >> Signed-off-by: Andrey Shinkevich >> --- >>tests/qemu-iotests/030 | 49 >>

[PATCH v2 1/3] iotests: Fix 173

2019-10-15 Thread Eric Blake
This test has been broken since 3.0. It used TEST_IMG to influence the name of a file created during _make_test_img, but commit 655ae6bb changed things so that the wrong file name is being created, which then caused _launch_qemu to fail. In the meantime, the set of events issued for the actions

Re: [PULL 0/2] Tracing patches

2019-10-15 Thread Philippe Mathieu-Daudé
On 10/15/19 2:24 PM, Peter Maydell wrote: On Mon, 14 Oct 2019 at 09:57, Stefan Hajnoczi wrote: The following changes since commit 98b2e3c9ab3abfe476a2b02f8f51813edb90e72d: Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging (2019-10-08 16:08:35 +0100)

Re: [RFC PATCH 23/23] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2019-10-15 Thread Eric Blake
On 10/15/19 10:23 AM, Alberto Garcia wrote: Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia --- +++ b/qapi/block-core.json @@ -85,6

Re: [RFC PATCH 00/23] Add subcluster allocation to qcow2

2019-10-15 Thread Eric Blake
On 10/15/19 10:23 AM, Alberto Garcia wrote: Hi, this series adds a new feature to the qcow2 on-disk format called "Extended L2 Entries", which allows us to do subcluster allocation. This cover letter explains the reasons behind this proposal, the changes to the on-disk format, test results and

[RFC PATCH 00/23] Add subcluster allocation to qcow2

2019-10-15 Thread Alberto Garcia
Hi, this series adds a new feature to the qcow2 on-disk format called "Extended L2 Entries", which allows us to do subcluster allocation. This cover letter explains the reasons behind this proposal, the changes to the on-disk format, test results and pending work. If you are curious you can also

[RFC PATCH 21/23] qcow2: Add subcluster support to handle_alloc_space()

2019-10-15 Thread Alberto Garcia
The bdrv_co_pwrite_zeroes() call here fills complete clusters with zeroes, but it can happen that some subclusters are not part of the write request or the copy-on-write. This patch makes sure that only the affected subclusters are overwritten. A potential improvement would be to also fill with

[RFC PATCH 14/23] qcow2: Add subcluster support to qcow2_get_cluster_offset()

2019-10-15 Thread Alberto Garcia
The logic of this function remains pretty much the same, except that it uses count_contiguous_subclusters(), which combines the logic of count_contiguous_clusters() / count_contiguous_clusters_unallocated() and checks individual subclusters. Signed-off-by: Alberto Garcia ---

[RFC PATCH 13/23] qcow2: Add subcluster support to calculate_l2_meta()

2019-10-15 Thread Alberto Garcia
If an image has subclusters then there are more copy-on-write scenarios that we need to consider. Let's say we have a write request from the middle of subcluster #3 until the end of the cluster: - If the cluster is new, then subclusters #0 to #3 from the old cluster must be copied into

[RFC PATCH 17/23] qcow2: Add subcluster support to check_refcounts_l2()

2019-10-15 Thread Alberto Garcia
Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-refcount.c | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git

[RFC PATCH 11/23] qcow2: Add qcow2_get_subcluster_type()

2019-10-15 Thread Alberto Garcia
This function returns the type of an individual subcluster. If an image does not have subclusters then this returns the exact same value as qcow2_get_cluster_type(). The information in standard and extended L2 entries is encoded in a slightly different way, but all existing QCow2ClusterType

[RFC PATCH 07/23] qcow2: Add subcluster-related fields to BDRVQcow2State

2019-10-15 Thread Alberto Garcia
This patch adds the following new fields to BDRVQcow2State: - subclusters_per_cluster: Number of subclusters in a cluster - subcluster_size: The size of each subcluster, in bytes - subcluster_bits: No. of bits so 1 << subcluster_bits = subcluster_size Images without subclusters are treated as if

[RFC PATCH 08/23] qcow2: Add offset_to_sc_index()

2019-10-15 Thread Alberto Garcia
For a given offset, return the subcluster number within its cluster (i.e. with 32 subclusters per cluster it returns a number between 0 and 31). Signed-off-by: Alberto Garcia --- block/qcow2.h | 5 + 1 file changed, 5 insertions(+) diff --git a/block/qcow2.h b/block/qcow2.h index

[RFC PATCH 18/23] qcow2: Add subcluster support to expand_zero_clusters_in_l1()

2019-10-15 Thread Alberto Garcia
Two changes are needed in order to add subcluster support to this function: deallocated clusters must have their bitmaps cleared, and expanded clusters must have all the "subcluster allocated" bits set. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 2 ++ 1 file changed, 2

[RFC PATCH 02/23] qcow2: Split cluster_needs_cow() out of count_cow_clusters()

2019-10-15 Thread Alberto Garcia
We are going to need it in other places. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 34 +++--- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c index fe2523ed66..f462e169c0 100644 ---

[PATCH v2 04/21] iotests: Filter refcount_order in 036

2019-10-15 Thread Max Reitz
This test can run just fine with other values for refcount_bits, so we should filter the value from qcow2.py's dump-header. In fact, we can filter everything but the feature bits and header extensions, because that is what the test is about. (036 currently ignores user-specified image options,

[PATCH v2 13/21] iotests: Avoid qemu-img create

2019-10-15 Thread Max Reitz
Use _make_test_img whenever possible. This way, we will not ignore user-specified image options. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/094 | 2 +- tests/qemu-iotests/111 | 3 +-- tests/qemu-iotests/123 | 2 +- tests/qemu-iotests/153 | 2 +-

[PATCH v2 14/21] iotests: Use _rm_test_img for deleting test images

2019-10-15 Thread Max Reitz
Just rm will not delete external data files. Use _rm_test_img every time we delete a test image. (In the process, clean up the indentation of every _cleanup() this patch touches.) ((Also, use quotes consistently. I am happy to see unquoted instances like "rm -rf $TEST_DIR/..." go.))

[PATCH v2 10/21] iotests: Replace IMGOPTS= by -o

2019-10-15 Thread Max Reitz
Tests should not overwrite all user-supplied image options, but only add to it (which will effectively overwrite conflicting values). Accomplish this by passing options to _make_test_img via -o instead of $IMGOPTS. For some tests, there is no functional change because they already only appended

[PATCH v2 20/21] iotests: Disable data_file where it cannot be used

2019-10-15 Thread Max Reitz
Signed-off-by: Max Reitz --- tests/qemu-iotests/007 | 5 +++-- tests/qemu-iotests/014 | 2 ++ tests/qemu-iotests/015 | 5 +++-- tests/qemu-iotests/026 | 5 - tests/qemu-iotests/029 | 5 +++-- tests/qemu-iotests/031 | 6 +++--- tests/qemu-iotests/036 | 5 +++-- tests/qemu-iotests/039 | 3 +++

Re: [PATCH v2 1/2] nbd: Don't send oversize strings

2019-10-15 Thread Eric Blake
On 10/11/19 2:32 AM, Vladimir Sementsov-Ogievskiy wrote: 11.10.2019 0:00, Eric Blake wrote: Qemu as server currently won't accept export names larger than 256 bytes, nor create dirty bitmap names longer than 1023 bytes, so most uses of qemu as client or server have no reason to get anywhere

[RFC PATCH 09/23] qcow2: Add l2_entry_size()

2019-10-15 Thread Alberto Garcia
qcow2 images with subclusters have 128-bit L2 entries. The first 64 bits contain the same information as traditional images and the last 64 bits form a bitmap with the status of each individual subcluster. Because of that we cannot assume that L2 entries are sizeof(uint64_t) anymore. This

[RFC PATCH 04/23] qcow2: Add get_l2_entry() and set_l2_entry()

2019-10-15 Thread Alberto Garcia
The size of an L2 entry is 64 bits, but if we want to have subclusters we need extended L2 entries. This means that we have to access L2 tables and slices differently depending on whether an image has extended L2 entries or not. This patch replaces all l2_slice[] accesses with calls to

[RFC PATCH 16/23] qcow2: Add subcluster support to discard_in_l2_slice()

2019-10-15 Thread Alberto Garcia
Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git

[PATCH v2 06/21] iotests: Drop compat=1.1 in 050

2019-10-15 Thread Max Reitz
IMGOPTS can never be empty for qcow2, because the check scripts adds compat=1.1 unless the user specified any compat option themselves. Thus, this block does not do anything and can be dropped. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/050 | 4 1 file

[PATCH v2 18/21] iotests: Make 137 work with data_file

2019-10-15 Thread Max Reitz
When using an external data file, there are no refcounts for data clusters. We thus have to adjust the corruption test in this patch to not be based around a data cluster allocation, but the L2 table allocation (L2 tables are still refcounted with external data files). Furthermore, we should not

[PATCH v2 21/21] iotests: Allow check -o data_file

2019-10-15 Thread Max Reitz
The problem with allowing the data_file option is that you want to use a different data file per image used in the test. Therefore, we need to allow patterns like -o data_file='$TEST_IMG.data_file'. Then, we need to filter it out from qemu-img map, qemu-img create, and remove the data file in

[RFC PATCH 10/23] qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()

2019-10-15 Thread Alberto Garcia
Extended L2 entries are 128-bit wide: 64 bits for the entry itself and 64 bits for the subcluster allocation bitmap. In order to support them correctly get/set_l2_entry() need to be updated so they take the entry width into account in order to calculate the correct offset. This patch also adds

[RFC PATCH 03/23] qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()

2019-10-15 Thread Alberto Garcia
When writing to a qcow2 file there are two functions that take a virtual offset and return a host offset, possibly allocating new clusters if necessary: - handle_copied() looks for normal data clusters that are already allocated and have a reference count of 1. In those clusters we

[RFC PATCH 01/23] qcow2: Add calculate_l2_meta()

2019-10-15 Thread Alberto Garcia
handle_alloc() creates a QCowL2Meta structure in order to update the image metadata and perform the necessary copy-on-write operations. This patch moves that code to a separate function so it can be used from other places. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 76

Re: [PULL 0/2] Tracing patches

2019-10-15 Thread Peter Maydell
On Tue, 15 Oct 2019 at 16:38, Philippe Mathieu-Daudé wrote: > > On 10/15/19 2:24 PM, Peter Maydell wrote: > > On Mon, 14 Oct 2019 at 09:57, Stefan Hajnoczi wrote: > >> > >> The following changes since commit > >> 98b2e3c9ab3abfe476a2b02f8f51813edb90e72d: > >> > >>Merge remote-tracking

[PATCH v2 12/21] iotests: Drop IMGOPTS use in 267

2019-10-15 Thread Max Reitz
Overwriting IMGOPTS means ignoring all user-supplied options, which is not what we want. Replace the current IMGOPTS use by a new BACKING_FILE variable. Signed-off-by: Max Reitz --- tests/qemu-iotests/267 | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git

[PATCH v2 19/21] iotests: Make 198 work with data_file

2019-10-15 Thread Max Reitz
We do not care about the json:{} filenames here, so we can just filter them out and thus make the test work both with and without external data files. Signed-off-by: Max Reitz --- tests/qemu-iotests/198 | 6 -- tests/qemu-iotests/198.out | 4 ++-- 2 files changed, 6 insertions(+), 4

[PATCH v2 17/21] iotests: Make 110 work with data_file

2019-10-15 Thread Max Reitz
The only difference is that the json:{} filename of the image looks different. We actually do not care about that filename in this test, we are only interested in (1) that there is a json:{} filename, and (2) whether the backing filename can be constructed. So just filter out the json:{} data,

[RFC PATCH 23/23] qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit

2019-10-15 Thread Alberto Garcia
Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia --- block/qcow2.c| 47 ++ block/qcow2.h

[RFC PATCH 20/23] qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()

2019-10-15 Thread Alberto Garcia
The L2 bitmap needs to be updated after each write to indicate what new subclusters are now allocated. This needs to happen even if the cluster was already allocated and the L2 entry was otherwise valid. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 16 1 file

[PATCH v2 07/21] iotests: Let _make_test_img parse its parameters

2019-10-15 Thread Max Reitz
This will allow us to add more options than just -b. Signed-off-by: Max Reitz Reviewed-by: Maxim Levitsky --- tests/qemu-iotests/common.rc | 28 1 file changed, 20 insertions(+), 8 deletions(-) diff --git a/tests/qemu-iotests/common.rc

[RFC PATCH 06/23] qcow2: Add dummy has_subclusters() function

2019-10-15 Thread Alberto Garcia
This function will be used by the qcow2 code to check if an image has subclusters or not. At the moment this simply returns false. Once all patches needed for subcluster support are ready then QEMU will be able to create and read images with subclusters and this function will return the actual

[RFC PATCH 12/23] qcow2: Handle QCOW2_CLUSTER_UNALLOCATED_SUBCLUSTER

2019-10-15 Thread Alberto Garcia
In the previous patch we added a new QCow2ClusterType named QCOW2_CLUSTER_UNALLOCATED_SUBCLUSTER. There is a couple of places where this new value needs to be handled, and that is what this patch does. Signed-off-by: Alberto Garcia --- block/qcow2.c | 10 +++--- 1 file changed, 7

[RFC PATCH 05/23] qcow2: Document the Extended L2 Entries feature

2019-10-15 Thread Alberto Garcia
Subcluster allocation in qcow2 is implemented by extending the existing L2 table entries and adding additional information to indicate the allocation status of each subcluster. This patch documents the changes to the qcow2 format and how they affect the calculation of the L2 cache size.

[RFC PATCH 22/23] qcow2: Restrict qcow2_co_pwrite_zeroes() to full clusters only

2019-10-15 Thread Alberto Garcia
Ideally it should be possible to zero individual subclusters using this function, but this is currently not implemented. Signed-off-by: Alberto Garcia --- block/qcow2.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/qcow2.c b/block/qcow2.c index c54278ab0b..2eb032aed7 100644

[RFC PATCH 19/23] qcow2: Fix offset calculation in handle_dependencies()

2019-10-15 Thread Alberto Garcia
l2meta_cow_start() and l2meta_cow_end() are not necessarily cluster-aligned if the image has subclusters, so update the calculation of old_start and old_end to guarantee that no two requests try to write on the same cluster. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 4 ++-- 1

[RFC PATCH 15/23] qcow2: Add subcluster support to zero_in_l2_slice()

2019-10-15 Thread Alberto Garcia
Setting the QCOW_OFLAG_ZERO bit of the L2 entry is forbidden if an image has subclusters. Instead, the individual 'all zeroes' bits must be used. Signed-off-by: Alberto Garcia --- block/qcow2-cluster.c | 14 ++ 1 file changed, 10 insertions(+), 4 deletions(-) diff --git

Re: [PATCH v2 1/2] nbd: Don't send oversize strings

2019-10-15 Thread Vladimir Sementsov-Ogievskiy
15.10.2019 18:07, Eric Blake wrote: > On 10/11/19 2:32 AM, Vladimir Sementsov-Ogievskiy wrote: >> 11.10.2019 0:00, Eric Blake wrote: >>> Qemu as server currently won't accept export names larger than 256 >>> bytes, nor create dirty bitmap names longer than 1023 bytes, so most >>> uses of qemu as

Re: [PULL 00/15] Block layer patches

2019-10-15 Thread Peter Maydell
On Mon, 14 Oct 2019 at 17:03, Kevin Wolf wrote: > > The following changes since commit 22dbfdecc3c52228d3489da3fe81da92b21197bf: > > Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20191010.0' > into staging (2019-10-14 15:09:08 +0100) > > are available in the Git repository

Re: [PATCH v2 00/20] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-10-15 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20191015103900.313928-1-...@irrelevant.dk/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally. === TEST SCRIPT BEGIN ===

Re: [PATCH 2/2] core: replace getpagesize() with qemu_real_host_page_size

2019-10-15 Thread Wei Yang
On Tue, Oct 15, 2019 at 02:45:15PM +0300, Yuval Shaia wrote: >On Sun, Oct 13, 2019 at 10:11:45AM +0800, Wei Yang wrote: >> There are three page size in qemu: >> >> real host page size >> host page size >> target page size >> >> All of them have dedicate variable to represent. For the last

Re: [PATCH v3 0/5] qcow2: advanced compression options

2019-10-15 Thread no-reply
Patchew URL: https://patchew.org/QEMU/1571163625-642312-1-git-send-email-andrey.shinkev...@virtuozzo.com/ Hi, This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

Re: [PATCH v2 00/21] iotests: Allow ./check -o data_file

2019-10-15 Thread no-reply
Patchew URL: https://patchew.org/QEMU/20191015142729.18123-1-mre...@redhat.com/ Hi, This series seems to have some coding style problems. See output below for more information: Subject: [PATCH v2 00/21] iotests: Allow ./check -o data_file Type: series Message-id:

Re: [PATCH 2/2] core: replace getpagesize() with qemu_real_host_page_size

2019-10-15 Thread Wei Yang
On Sun, Oct 13, 2019 at 08:28:41PM +1100, David Gibson wrote: >On Sun, Oct 13, 2019 at 10:11:45AM +0800, Wei Yang wrote: >> There are three page size in qemu: >> >> real host page size >> host page size >> target page size >> >> All of them have dedicate variable to represent. For the last

Re: [PULL 1/1] test-bdrv-drain: fix iothread_join() hang

2019-10-15 Thread Stefan Hajnoczi
On Mon, Oct 14, 2019 at 01:11:41PM +0200, Paolo Bonzini wrote: > On 14/10/19 10:52, Stefan Hajnoczi wrote: > > tests/test-bdrv-drain can hang in tests/iothread.c:iothread_run(): > > > > while (!atomic_read(>stopping)) { > > aio_poll(iothread->ctx, true); > > } > > > > The

Re: [PULL 01/19] util/hbitmap: strict hbitmap_reset

2019-10-15 Thread Kevin Wolf
Am 14.10.2019 um 20:10 hat John Snow geschrieben: > > > On 10/11/19 7:18 PM, John Snow wrote: > > > > > > On 10/11/19 5:48 PM, Eric Blake wrote: > >> On 10/11/19 4:25 PM, John Snow wrote: > >>> From: Vladimir Sementsov-Ogievskiy > >>> > >>> hbitmap_reset has an unobvious property: it rounds

Re: [PULL 1/2] trace: add --group=all to tracing.txt

2019-10-15 Thread Stefan Hajnoczi
On Mon, Oct 14, 2019 at 11:08:25AM +0200, Philippe Mathieu-Daudé wrote: > Hi Stefan, > > On 10/14/19 10:57 AM, Stefan Hajnoczi wrote: > > tracetool needs to know the group name ("all", "root", or a specific > > subdirectory). Also remove the stdin redirection because tracetool.py > > needs the

[PATCH v2 07/20] nvme: refactor device realization

2019-10-15 Thread Klaus Jensen
This patch splits up nvme_realize into multiple individual functions, each initializing a different subset of the device. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 176 +++- hw/block/nvme.h | 22 ++ 2 files changed, 135 insertions(+), 63

[PATCH v2 00/20] nvme: support NVMe v1.3d, SGLs and multiple namespaces

2019-10-15 Thread Klaus Jensen
Hi, (Quick note to Fam): most of this series is irrelevant to you as the maintainer of the nvme block driver, but patch "nvme: add support for scatter gather lists" touches block/nvme.c due to changes in the shared NvmeCmd struct. Anyway, v2 comes with a good bunch of changes. Compared to v1[1],

[PATCH v2 06/20] nvme: add support for the abort command

2019-10-15 Thread Klaus Jensen
Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.1 ("Abort command"). The Abort command is a best effort command; for now, the device always fails to abort the given command. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 16 1 file

[PATCH v2 02/20] nvme: move device parameters to separate struct

2019-10-15 Thread Klaus Jensen
Move device configuration parameters to separate struct to make it explicit what is configurable and what is set internally. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 44 ++-- hw/block/nvme.h | 16 +--- 2 files changed, 35

[PATCH v2 03/20] nvme: add missing fields in the identify controller data structure

2019-10-15 Thread Klaus Jensen
Not used by the device model but added for completeness. See NVM Express 1.2.1, Section 5.11 ("Identify command"), Figure 90. Signed-off-by: Klaus Jensen --- include/block/nvme.h | 34 +- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git

[PATCH v2 04/20] nvme: populate the mandatory subnqn and ver fields

2019-10-15 Thread Klaus Jensen
Required for compliance with NVMe revision 1.2.1 or later. See NVM Express 1.2.1, Section 5.11 ("Identify command"), Figure 90 and Section 7.9 ("NVMe Qualified Names"). This also bumps the supported version to 1.2.1. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 11 --- 1 file

[PATCH v2 10/20] nvme: add logging to error information log page

2019-10-15 Thread Klaus Jensen
This adds the nvme_set_error_page function which allows errors to be written to the error information log page. The functionality is largely unused in the device, but with this in place we can at least try to push new contributions to use it. NOTE: In violation of the specification the Error

[PATCH v2 09/20] nvme: add support for the asynchronous event request command

2019-10-15 Thread Klaus Jensen
Required for compliance with NVMe revision 1.2.1. See NVM Express 1.2.1, Section 5.2 ("Asynchronous Event Request command"). Mostly imported from Keith's qemu-nvme tree. Modified to not enqueue events if something of the same type is already queued (but not cleared by the host). Signed-off-by:

[PATCH v2 08/20] nvme: add support for the get log page command

2019-10-15 Thread Klaus Jensen
Add support for the Get Log Page command and basic implementations of the mandatory Error Information, SMART/Health Information and Firmware Slot Information log pages. In violation of the specification, the SMART/Health Information log page does not persist information over the lifetime of the

[PATCH v2 11/20] nvme: add missing mandatory features

2019-10-15 Thread Klaus Jensen
Add support for returning a resonable response to Get/Set Features of mandatory features. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 51 --- hw/block/trace-events | 2 ++ include/block/nvme.h | 3 ++- 3 files changed, 52 insertions(+), 4

[PATCH v2 15/20] nvme: add support for scatter gather lists

2019-10-15 Thread Klaus Jensen
For now, support the Data Block, Segment and Last Segment descriptor types. See NVM Express 1.3d, Section 4.4 ("Scatter Gather List (SGL)"). Signed-off-by: Klaus Jensen --- block/nvme.c | 18 +- hw/block/nvme.c | 380 --

Re: [PULL 0/1] Block patches

2019-10-15 Thread Peter Maydell
On Mon, 14 Oct 2019 at 09:52, Stefan Hajnoczi wrote: > > The following changes since commit 98b2e3c9ab3abfe476a2b02f8f51813edb90e72d: > > Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' > into staging (2019-10-08 16:08:35 +0100) > > are available in the Git repository

[PATCH v2 01/20] nvme: remove superfluous breaks

2019-10-15 Thread Klaus Jensen
These break statements was left over when commit 3036a626e9ef ("nvme: add Get/Set Feature Timestamp support") was merged. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 4 1 file changed, 4 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c index 12d825425016..c06e3ca31905

[PATCH v2 05/20] nvme: allow completion queues in the cmb

2019-10-15 Thread Klaus Jensen
Allow completion queues in the controller memory buffer. This also inlines the nvme_addr_{read,write} functions and adds an nvme_addr_is_cmb helper. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 38 +- 1 file changed, 29 insertions(+), 9 deletions(-)

[PATCH v2 12/20] nvme: bump supported specification version to 1.3

2019-10-15 Thread Klaus Jensen
Add the new Namespace Identification Descriptor List (CNS 03h) and track creation of queues to enable the controller to return Command Sequence Error if Set Features is called for Number of Queues after any queues have been created. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 82

[PATCH v2 18/20] nvme: remove redundant NvmeCmd pointer parameter

2019-10-15 Thread Klaus Jensen
The command struct is available in the NvmeRequest that we generally pass around anyway. Signed-off-by: Klaus Jensen --- hw/block/nvme.c | 219 +++- 1 file changed, 106 insertions(+), 113 deletions(-) diff --git a/hw/block/nvme.c b/hw/block/nvme.c

[PATCH v2 14/20] nvme: allow multiple aios per command

2019-10-15 Thread Klaus Jensen
This refactors how the device issues asynchronous block backend requests. The NvmeRequest now holds a queue of NvmeAIOs that are associated with the command. This allows multiple aios to be issued for a command. Only when all requests have been completed will the device post a completion queue

[PATCH v2 20/20] nvme: handle dma errors

2019-10-15 Thread Klaus Jensen
Handling DMA errors gracefully is required for the device to pass the block/011 test ("disable PCI device while doing I/O") in the blktests suite. With this patch the device passes the test by retrying "critical" transfers (posting of completion entries and processing of submission queue

[PATCH v2 16/20] nvme: support multiple namespaces

2019-10-15 Thread Klaus Jensen
This adds support for multiple namespaces by introducing a new 'nvme-ns' device model. The nvme device creates a bus named from the device name ('id'). The nvme-ns devices then connect to this and registers themselves with the nvme device. This changes how an nvme device is created. Example with

[PATCH v2 13/20] nvme: refactor prp mapping

2019-10-15 Thread Klaus Jensen
Instead of handling both QSGs and IOVs in multiple places, simply use QSGs everywhere by assuming that the request does not involve the controller memory buffer (CMB). If the request is found to involve the CMB, convert the QSG to an IOV and issue the I/O. The QSG is converted to an IOV by the dma

[PATCH v2 19/20] nvme: make lba data size configurable

2019-10-15 Thread Klaus Jensen
Signed-off-by: Klaus Jensen --- hw/block/nvme-ns.c | 2 +- hw/block/nvme-ns.h | 4 +++- hw/block/nvme.c| 1 + 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/hw/block/nvme-ns.c b/hw/block/nvme-ns.c index aa76bb63ef45..70ff622a5729 100644 --- a/hw/block/nvme-ns.c +++

  1   2   >