I have clearly been lax in checking my coding style...
I'll fix these, but not until review.
On 7/3/19 9:50 PM, no-re...@patchew.org wrote:
> Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/
>
>
>
> Hi,
>
> This series seems to have some coding style problems.
Patchew URL: https://patchew.org/QEMU/20190703224707.12437-1-ebl...@redhat.com/
Hi,
This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make
Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/
Hi,
This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make
Patchew URL: https://patchew.org/QEMU/20190703215542.16123-1-js...@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Subject: [Qemu-devel] [PATCH v2 00/18] bitmaps: introduce 'bitmap' sync mode
Message-id:
Although you generally won't use encryption with a Unix socket (after
all, everything is local, so why waste the CPU power), there are
situations in testsuites where Unix sockets are much nicer than TCP
sockets. Since nbdkit allows encryption over both types of sockets,
it makes sense for
run_job can cancel pending jobs to simulate failure. This lets us use
the pending callback to issue test commands while the job is open, but
then still have the job fail in the end.
Signed-off-by: John Snow
---
tests/qemu-iotests/iotests.py | 22 --
1 file changed, 20
Add a public interface for get. While we're at it,
rename "bdrv_get_dirty_bitmap_locked" to "bdrv_dirty_bitmap_get_locked".
(There are more functions to rename to the bdrv_dirty_bitmap_VERB form,
but they will wait until the conclusion of this series.)
Signed-off-by: John Snow
---
Signed-off-by: John Snow
---
util/hbitmap.c | 36 +++-
1 file changed, 35 insertions(+), 1 deletion(-)
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 3b6acae42b..306bc4876d 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -777,7 +777,27 @@ void
With the "never" sync policy, we actually can utilize readonly bitmaps
now. Loosen the check at the QMP level, and tighten it based on
provided arguments down at the job creation level instead.
Signed-off-by: John Snow
---
block/backup.c | 6 ++
blockdev.c | 2 +-
2 files changed, 7
Nobody calls the function like this currently, but we neither prohibit
or cope with this behavior. I decided to make the function cope with it.
Signed-off-by: John Snow
---
util/hbitmap.c | 13 ++---
1 file changed, 10 insertions(+), 3 deletions(-)
diff --git a/util/hbitmap.c
We don't need or want a new sync mode for simple differences in
semantics. Create a new mode simply named "BITMAP" that is designed to
make use of the new Bitmap Sync Mode field.
Because the only bitmap mode is 'conditional', this adds no new
functionality to the backup job (yet). The old
This adds a "never" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, we never update the bitmap. This can be used
to perform differential backups, or simply to avoid the job modifying a
bitmap.
Signed-off-by: John Snow
---
block/backup.c | 5 +++--
Signed-off-by: John Snow
---
tests/qemu-iotests/257 | 409 +++
tests/qemu-iotests/257.out | 2199
tests/qemu-iotests/group |1 +
3 files changed, 2609 insertions(+)
create mode 100755 tests/qemu-iotests/257
create mode 100644
This adds an "always" policy for bitmap synchronization. Regardless of if
the job succeeds or fails, the bitmap is *always* synchronized. This means
that for backups that fail part-way through, the bitmap retains a record of
which sectors need to be copied out to accomplish a new backup using the
Because the new-style python tests don't use the iotests.main() test
launcher, we don't turn on the debugger logging for these scripts
when invoked via ./check -d.
Refactor the launcher shim into new and old style shims so that they
share environmental configuration.
Two cleanup notes: debug was
Depending on what a user is trying to accomplish, there might be a few
bitmap cleanup actions that occur when an operation is finished that
could be useful.
I am proposing three:
- NEVER: The bitmap is never synchronized against what was copied.
- ALWAYS: The bitmap is always synchronized, even
Seems that it comes up enough.
Signed-off-by: John Snow
---
tests/qemu-iotests/040| 6 +-
tests/qemu-iotests/093| 6 ++
tests/qemu-iotests/139| 7 ++-
tests/qemu-iotests/238| 5 +
tests/qemu-iotests/iotests.py | 4
5 files changed, 10
drive-backup and blockdev-backup have an awful lot of things in common
that are the same. Let's fix that.
I don't deduplicate 'target', because the semantics actually did change
between each structure. Leave that one alone so it can be documented
separately.
Signed-off-by: John Snow
---
Signed-off-by: John Snow
---
blockdev.c | 73 +++---
1 file changed, 3 insertions(+), 70 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 5fd663a7e5..cad300d526 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3638,82 +3638,15 @@ XDbgBlockGraph
I'm surprised it didn't come up sooner, but sometimes we have a +busy
bitmap as a source. This is dangerous from the QMP API, but if we are
the owner that marked the bitmap busy, it's safe to merge it using it as
a read only source.
It is not safe in the general case to allow users to read from
This simplifies some interface matters; namely the initialization and
(later) merging the manifest back into the sync_bitmap if it was
provided.
Signed-off-by: John Snow
---
block/backup.c | 76 --
1 file changed, 37 insertions(+), 39 deletions(-)
Use "FilePaths" instead of "FilePath" to request multiple files be
cleaned up after we leave that object's scope.
This is not crucial; but it saves a little typing.
Signed-off-by: John Snow
---
tests/qemu-iotests/iotests.py | 33 ++---
1 file changed, 22
This series adds a new "BITMAP" sync mode that is meant to replace the
existing "INCREMENTAL" sync mode.
This mode can have its behavior modified by issuing any of three bitmap sync
modes, passed as arguments to the job.
The three bitmap sync modes are:
- CONDITIONAL: This is an alias for the
Create a common core that comprises the actual meat of what the backup API
boundary needs to do, and then switch drive-backup to use it.
Questions:
- do_drive_backup now acquires and releases the aio_context in addition
to do_backup_common doing the same. Can I drop this from drive_backup?
Patchew URL: https://patchew.org/QEMU/20190703155944.9637-1-mlevi...@redhat.com/
Hi,
This series failed the asan build test. Please find the testing commands and
their output below. If you have Docker installed, you can probably reproduce it
locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make
On 7/3/19 3:30 PM, Max Reitz wrote:
> On 01.07.19 22:13, John Snow wrote:
>> It is used to do transactional movement of the bitmap (which is
>> possible in conjunction with merge command). Transactional bitmap
>> movement is needed in scenarios with external snapshot, when we don't
>> want to
On 03.07.19 21:30, Max Reitz wrote:
> On 01.07.19 22:13, John Snow wrote:
>> It is used to do transactional movement of the bitmap (which is
>> possible in conjunction with merge command). Transactional bitmap
>> movement is needed in scenarios with external snapshot, when we don't
>> want to
On 01.07.19 22:13, John Snow wrote:
> It is used to do transactional movement of the bitmap (which is
> possible in conjunction with merge command). Transactional bitmap
> movement is needed in scenarios with external snapshot, when we don't
> want to leave copy of the bitmap in the base image.
>
Currently, 030 just compares the error class, which does not say
anything.
Before HEAD^ added throttling to test_overlapping_4, that test actually
usually failed because node2 was already gone, not because it was the
commit and stream job were not allowed to overlap.
Prevent such problems in the
Currently, TestParallelOps in 030 creates images that are too small for
job throttling to be effective. This is reflected by the fact that it
never undoes the throttling.
Increase the image size and undo the throttling when the job should be
completed. Also, add throttling in
bdrv_change_backing_file() can result in yields. Therefore, @base may
no longer be the the backing_bs() of s->bottom afterwards.
Just swap the order of the two calls to fix this.
Signed-off-by: Max Reitz
---
block/stream.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
On 28/06/2019 16:46, Andrey Shinkevich wrote:
> The Valgrind tool reports about the uninitialised buffer 'buf'
> instantiated on the stack of the function guess_disk_lchs().
> It is revealed in the tests 051, 186, 227 and 240.
> Pass 'read-zeroes=on' to the null block driver in the mentioned
We recently removed the dependency of the stream job on its base node.
That makes it OK to use a commit filter node there. Test that.
Signed-off-by: Max Reitz
Tested-by: Andrey Shinkevich
Reviewed-by: Alberto Garcia
---
tests/qemu-iotests/030 | 25 +
BDS.inherits_from does not always point to an immediate parent node.
When launching a block job with a filter node, for example, the node
directly below the filter will not point to the filter, but keep its old
pointee (above the filter).
If that pointee goes away while the job is still running,
unittest-style tests generally do not use the log file, but VM.run_job()
can still be useful to them. Add a parameter to it that hides its
output from the log file.
Signed-off-by: Max Reitz
---
tests/qemu-iotests/iotests.py | 18 +-
1 file changed, 13 insertions(+), 5
This is a v2 to “block: Add BDS.never_freeze”.
It depends on my “block: Delay poll when ending drained sections”
series:
Depends-on: <20190619152603.5937-1-mre...@redhat.com>
It turned out that if you run 030 (or just the new test_overlapping_5
case) sufficiently often, it breaks; which is why
Currently, bdrv_replace_child_noperm() undrains the parent until it is
completely undrained, then re-drains it after attaching the new child
node.
This is a problem with bdrv_drop_intermediate(): We want to keep the
whole subtree drained, including parents, while the operation is
under way.
bdrv_drop_intermediate() calls BdrvChildRole.update_filename(). That
may poll, thus changing the graph, which potentially breaks the
QLIST_FOREACH_SAFE() loop.
Just keep the whole subtree drained. This is probably the right thing
to do anyway (dropping nodes while the subtree is not drained
We already have 030 for that in general, but this tests very specific
cases of both jobs finishing concurrently.
Signed-off-by: Max Reitz
---
tests/qemu-iotests/258 | 163 +
tests/qemu-iotests/258.out | 33
tests/qemu-iotests/group | 1 +
3
This tests that the stream job exits cleanly (without abort) when the
top node is read-only and cannot be reopened read/write.
Signed-off-by: Max Reitz
---
tests/qemu-iotests/030 | 29 -
tests/qemu-iotests/030.out | 4 ++--
2 files changed, 30 insertions(+), 3
As of commit c624b015bf14fe01f1e6452a36e63b3ea1ae4998, the stream job
only freezes the chain until the overlay of the base node. The error
path must consider this.
Fixes: c624b015bf14fe01f1e6452a36e63b3ea1ae4998
Signed-off-by: Max Reitz
---
block/stream.c | 2 +-
1 file changed, 1
The commit and the mirror block job must be able to drop their filter
node at any point. However, this will not be possible if any of the
BdrvChild links to them is frozen. Therefore, we need to prevent them
from ever becoming frozen.
Signed-off-by: Max Reitz
Reviewed-by: Andrey Shinkevich
> On Jul 3, 2019, at 12:02, Philippe Mathieu-Daudé wrote:
>
> On 7/3/19 5:52 PM, Stephen Checkoway wrote:
>>
>>
>>> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote:
>>>
>>> Parallel NOR flashes are limited to 16-bit bus accesses.
>>
>> I don't think this is correct. The CFI spec
On 7/3/19 5:52 PM, Stephen Checkoway wrote:
>
>
>> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote:
>>
>> Parallel NOR flashes are limited to 16-bit bus accesses.
>
> I don't think this is correct. The CFI spec defines an x32 interface for
> parallel NOR. CFI addresses 0x28 and 0x29
On 7/3/19 6:20 PM, Stephen Checkoway wrote:
>> On Jul 3, 2019, at 12:02, Philippe Mathieu-Daudé wrote:
>> On 7/3/19 5:52 PM, Stephen Checkoway wrote:
>>>
>>>
On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote:
Parallel NOR flashes are limited to 16-bit bus accesses.
>>>
>>> I
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 81 ++
block/trace-events | 2 ++
2 files changed, 83 insertions(+)
diff --git a/block/nvme.c b/block/nvme.c
index 02e0846643..96a715dcc1 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -111,6
Am 03.07.2019 um 16:36 hat Eric Blake geschrieben:
> On 7/3/19 6:00 AM, Denis Plotnikov wrote:
> > zstd significantly reduces cluster compression time.
> > It provides better compression performance maintaining
> > the same level of compression ratio in comparison with
> > zlib, which, by the
Currently the driver hardcodes the sector size to 512,
and doesn't check the underlying device. Fix that.
Also fail if underlying nvme device is formatted with metadata
as this needs special support.
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 45
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 81 ++
block/trace-events | 2 ++
2 files changed, 83 insertions(+)
diff --git a/block/nvme.c b/block/nvme.c
index 02e0846643..f8bcf1ffb6 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -111,6
Compared to last submission, this series adds another patch,
which implements support for image creation over the nvme drive like that:
qemu-img create -f qcow2 nvme://:06:00.0/1 10G -o preallocation=metadata
I also addressed the review comments.
Best regards,
Maxim Levitsky
Maxim
Tesed on a nvme device like that:
# create preallocated qcow2 image
$ qemu-img create -f qcow2 nvme://:06:00.0/1 10G -o preallocation=metadata
Formatting 'nvme://:06:00.0/1', fmt=qcow2 size=10737418240
cluster_size=65536 preallocation=metadata lazy_refcounts=off refcount_bits=16
#
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 69 +++-
block/trace-events | 1 +
include/block/nvme.h | 19 +++-
3 files changed, 87 insertions(+), 2 deletions(-)
diff --git a/block/nvme.c b/block/nvme.c
index 152d27b07f..02e0846643
Fix the math involving non standard doorbell stride
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/nvme.c b/block/nvme.c
index 6d4e7f3d83..52798081b2 100644
--- a/block/nvme.c
+++ b/block/nvme.c
@@ -217,7 +217,7 @@ static
On 03.07.2019 18:46, Kevin Wolf wrote:
> Am 03.07.2019 um 17:01 hat Denis Plotnikov geschrieben:
>> On 03.07.2019 17:14, Eric Blake wrote:
>>> On 7/3/19 6:00 AM, Denis Plotnikov wrote:
diff --git a/block/qcow2.c b/block/qcow2.c
index 3ace3b2209..921eb67b80 100644
---
Completion entries are meant to be only read by the host and written by the
device.
The driver is supposed to scan the completions from the last point where it
left,
and until it sees a completion with non flipped phase bit.
Signed-off-by: Maxim Levitsky
---
block/nvme.c | 5 +
1 file
> On Jul 1, 2019, at 20:59, Philippe Mathieu-Daudé wrote:
>
> Parallel NOR flashes are limited to 16-bit bus accesses.
I don't think this is correct. The CFI spec defines an x32 interface for
parallel NOR. CFI addresses 0x28 and 0x29 specify the interface and value 3 is
x32 and value 5 is
Am 03.07.2019 um 17:01 hat Denis Plotnikov geschrieben:
> On 03.07.2019 17:14, Eric Blake wrote:
> > On 7/3/19 6:00 AM, Denis Plotnikov wrote:
> >> diff --git a/block/qcow2.c b/block/qcow2.c
> >> index 3ace3b2209..921eb67b80 100644
> >> --- a/block/qcow2.c
> >> +++ b/block/qcow2.c
> >> @@ -1202,6
On 03.07.2019 18:13, Eric Blake wrote:
> On 7/3/19 10:01 AM, Denis Plotnikov wrote:
>
+ * with QCOW2_COMPRESSION_TYPE_ZLIB the corresponding incompatible
+ * feature flag must be absent, with other compression types the
+ * incompatible feature flag must be set
>>> Is
On Wed, 2019-07-03 at 09:50 -0500, Eric Blake wrote:
> On 6/30/19 10:08 AM, Maxim Levitsky wrote:
> > Regular block devices (/dev/sda*, /dev/nvme*, etc) interface is not limited
>
> The regular block device interface is
>
> or
>
> Regular block devices interfaces are
>
> > by the underlying
On 7/3/19 10:01 AM, Denis Plotnikov wrote:
>>> + * with QCOW2_COMPRESSION_TYPE_ZLIB the corresponding incompatible
>>> + * feature flag must be absent, with other compression types the
>>> + * incompatible feature flag must be set
>> Is there a strong reason for forbid the
On 03.07.2019 17:36, Eric Blake wrote:
> On 7/3/19 6:00 AM, Denis Plotnikov wrote:
>> zstd significantly reduces cluster compression time.
>> It provides better compression performance maintaining
>> the same level of compression ratio in comparison with
>> zlib, which, by the moment, has been
On 03.07.2019 17:14, Eric Blake wrote:
> On 7/3/19 6:00 AM, Denis Plotnikov wrote:
>> The patch adds some preparation parts for incompatible compression type
>> feature to QCOW2 header that indicates that*all* compressed clusters
>> must be (de)compressed using a certain compression type.
>>
>>
On 6/30/19 10:08 AM, Maxim Levitsky wrote:
> Regular block devices (/dev/sda*, /dev/nvme*, etc) interface is not limited
The regular block device interface is
or
Regular block devices interfaces are
> by the underlying storage limits, but rather the kernel block layer
> takes care to split the
On 7/3/19 4:52 AM, Stefan Hajnoczi wrote:
> On Sun, Jun 30, 2019 at 06:08:54PM +0300, Maxim Levitsky wrote:
>> It looks like Linux block devices, even in O_DIRECT mode don't have any user
>> visible
>> limit on transfer size / number of segments, which underlying block device
>> can have.
>> The
On 7/3/19 6:00 AM, Denis Plotnikov wrote:
> zstd significantly reduces cluster compression time.
> It provides better compression performance maintaining
> the same level of compression ratio in comparison with
> zlib, which, by the moment, has been the only compression
> method available.
>
>
On 7/3/19 6:00 AM, Denis Plotnikov wrote:
> The patch adds some preparation parts for incompatible compression type
> feature to QCOW2 header that indicates that *all* compressed clusters
> must be (de)compressed using a certain compression type.
>
> It is implied that the compression type is set
change log:
v1:
* extend qcow2 header instead of adding a new incompatible extension header
specification re-written accordingly
* enable zstd compression via config
* fix zstd (de)compression functions
* fix comments/description
* fix function naming
---
The goal of
zstd significantly reduces cluster compression time.
It provides better compression performance maintaining
the same level of compression ratio in comparison with
zlib, which, by the moment, has been the only compression
method available.
The performance test results:
Test compresses and
The patch adds some preparation parts for incompatible compression type
feature to QCOW2 header that indicates that *all* compressed clusters
must be (de)compressed using a certain compression type.
It is implied that the compression type is set on the image creation and
can be changed only later
The patch allow to process image compression type defined
in the image header and choose an appropriate method for
image clusters (de)compression.
Signed-off-by: Denis Plotnikov
---
block/qcow2.c | 93 ---
1 file changed, 73 insertions(+), 20
On Sun, Jun 30, 2019 at 06:08:54PM +0300, Maxim Levitsky wrote:
> It looks like Linux block devices, even in O_DIRECT mode don't have any user
> visible
> limit on transfer size / number of segments, which underlying block device
> can have.
> The block layer takes care of enforcing these limits
On Tue, Jul 02, 2019 at 11:09:14AM -0400, Jason Dillaman wrote:
> On Fri, Jun 28, 2019 at 4:59 AM Stefano Garzarella
> wrote:
> >
> > On Thu, Jun 27, 2019 at 03:43:04PM -0400, Jason Dillaman wrote:
> > > On Thu, Jun 27, 2019 at 1:24 PM John Snow wrote:
> > > > On 6/27/19 4:48 AM, Stefano
I apologize for replying so late.
Stefan Hajnoczi writes:
> On Tue, Jun 11, 2019 at 03:36:53PM +0800, Fam Zheng wrote:
>> On Mon, 06/10 19:18, Aarushi Mehta wrote:
>> > Option only enumerates for hosts that support it.
>> >
>> > Signed-off-by: Aarushi Mehta
>> > Reviewed-by: Stefan Hajnoczi
73 matches
Mail list logo