On Wed, 05/13 13:17, Wen Congyang wrote:
> On 05/13/2015 11:11 AM, Fam Zheng wrote:
> > Before, we only yield after initializing dirty bitmap, where the QMP
> > command would return. That may take very long, and guest IO will be
> > blocked.
>
> Do you have such case to reproduce it? If the disk i
Test zero write in byte range 512~1024 for 4k alignment.
Signed-off-by: Fam Zheng
Reviewed-by: Stefan Hajnoczi
Reviewed-by: Kevin Wolf
---
tests/qemu-iotests/033 | 13 +
tests/qemu-iotests/033.out | 30 ++
2 files changed, 43 insertions(+)
diff --gi
On 05/13/2015 11:11 AM, Fam Zheng wrote:
> Before, we only yield after initializing dirty bitmap, where the QMP
> command would return. That may take very long, and guest IO will be
> blocked.
Do you have such case to reproduce it? If the disk image is too larger,
and I think qemu doesn't cache al
For zero write, callers pass in NULL qiov (qemu-io "write -z" or
scsi-disk "write same").
Commit fc3959e466 fixed bdrv_co_write_zeroes which is the common case
for this bug, but it still exists in bdrv_aio_write_zeroes. A simpler
fix would be in bdrv_co_do_pwritev which is the NULL dereference poi
v7: Add Kevin's rev-by in patch 1 and 3.
Address Stefan's and Kevin's comments on patch 2:
- Don't duplicate tracked_request_begin and tracked_request_end;
- Don't forget to remove debug printf;
- Call qemu_vfree unconditionally;
- Don't serialize aligned part of the zero write
This reverts commit fc3959e4669a1c2149b91ccb05101cfc7ae1fc05.
The core write code already handles the case, so remove this
duplication.
Because commit 61007b316 moved the touched code from block.c to
block/io.c, the change is manually reverted.
Signed-off-by: Fam Zheng
Reviewed-by: Stefan Hajno
On Tue, 05/12 13:52, Kevin Wolf wrote:
> Am 12.05.2015 um 08:09 hat Fam Zheng geschrieben:
> > For zero write, callers pass in NULL qiov (qemu-io "write -z" or
> > scsi-disk "write same").
> >
> > Commit fc3959e466 fixed bdrv_co_write_zeroes which is the common case
> > for this bug, but it still
On Tue, 05/12 13:18, Stefan Hajnoczi wrote:
> On Tue, May 12, 2015 at 02:09:31PM +0800, Fam Zheng wrote:
> > +static int coroutine_fn bdrv_co_do_zero_pwritev(BlockDriverState *bs,
> > +int64_t offset,
> > +
Before, we only yield after initializing dirty bitmap, where the QMP
command would return. That may take very long, and guest IO will be
blocked.
Add sleep points like the later mirror iterations.
Signed-off-by: Fam Zheng
---
block/mirror.c | 13 -
1 file changed, 12 insertions(+),
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> The code already special-cased "node-name", which is currently the only
> option passed in the QDict that isn't driver-specific. Generalise the
> code to take all general block layer options into consideration.
>
> Signed-off-by: Kevin Wolf
> ---
> blo
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> For updating the cache sizes or disabling lazy refcounts there is a bit
> more to do than just changing the variables, but otherwise we're all set
> for changing options during bdrv_reopen().
>
> Just implement the missing pieces and hook the functions u
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> Before we can allow updating options at runtime with bdrv_reopen(), we
> need to split the function into prepare/commit/abort parts.
>
> Signed-off-by: Kevin Wolf
> ---
> block/qcow2.c | 101
> ++
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf
> ---
> block/qcow2.c | 12 ++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
Reviewed-by: Eric Blake
--
Eric Blake eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> On return, either all new options should be applied to BDRVQcowState (on
> success), or all of the old setting should be preserved (on failure).
s/setting/settings/
>
> Signed-off-by: Kevin Wolf
> ---
> block/qcow2.c | 52
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> With this commit, the handling of driver-specific options in
> qcow2_open() is completely separated out into qcow2_update_options().
>
> Signed-off-by: Kevin Wolf
> ---
> block/qcow2.c | 109
> +-
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> qcow2_update_options() only updates some variables in BDRVQcowState and
> doesn't really depend on other parts of it being initialised yet, so it
> can be moved so that it immediately follows the other half of option
> handling code in qcow2_open().
>
>
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> Eventually we want to be able to change options at runtime. As a first
> step towards that goal, separate some option handling code from the
> general initialisation code in qcow2_open().
>
> Signed-off-by: Kevin Wolf
> ---
> block/qcow2.c | 135
> +++
On 05/12/2015 03:52 PM, Eric Blake wrote:
> On 05/12/2015 01:06 PM, John Snow wrote:
tests/qemu-iotests/131 | 69
++
tests/qemu-iotests/131.out | 46
+++
>
>>
>> Fam Zheng already has a patch on-list
On 05/12/2015 01:06 PM, John Snow wrote:
>>> tests/qemu-iotests/131 | 69
>>> ++
>>> tests/qemu-iotests/131.out | 46 +++
>
> Fam Zheng already has a patch on-list that uses test 131, and I think
> his patch was submitted
On Fri, May 08, 2015 at 07:21:37PM +0200, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf
> ---
> blockdev.c| 24
> include/block/block.h | 8
> 2 files changed, 20 insertions(+), 12 deletions(-)
>
> diff --git a/blockdev.c b/blockdev.c
> index 5eaf77
On Fri, May 08, 2015 at 07:21:36PM +0200, Kevin Wolf wrote:
> Besides standardising on a single interface for opening child nodes,
> this patch allows the user to specify options to individual extent
> nodes. Overriding file names isn't possible with this yet, so it's of
> limited usefulness, but s
On 05/12/2015 02:35 PM, Eric Blake wrote:
> On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
>> Add a simple test case for qemu-iotests that covers read/write
>> with encrypted qcow2 files.
>>
>> Signed-off-by: Daniel P. Berrange ---
>> tests/qemu-iotests/131 | 69
>>
On Fri, May 08, 2015 at 07:21:35PM +0200, Kevin Wolf wrote:
> Besides standardising on a single interface for opening child nodes,
> this simplifies the .bdrv_open() implementation of the quorum block
> driver by using block layer functionality for handling BlockdevRefs.
>
> Signed-off-by: Kevin W
On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
> Add a simple test case for qemu-iotests that covers read/write
> with encrypted qcow2 files.
>
> Signed-off-by: Daniel P. Berrange
> ---
> tests/qemu-iotests/131 | 69
> ++
> tests/qemu-iotests/1
On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
> The qemu-io tool does not check if the image is encrypted so
> historically would silently corrupt the sectors by writing
> plain text data into them instead of cipher text. The earlier
> commit turns this mistake into a fatal abort, so check for
On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
> The qemu_read_password() method looks for \r to terminate the
> reading of the a password. This is what will be seen when
> reading the password from a TTY. When scripting though, it is
> useful to be able to send the password via a pipe, in which
On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
> The qemu-img.c file has a read_password() method impl that is
> used to prompt for passwords on the console, with impls for
> POSIX and Windows. This will be needed by qemu-io.c too, so
> move it into the QEMU osdep/oslib files where it can be sha
On 05/12/2015 10:09 AM, Daniel P. Berrange wrote:
> When a qcow[2] file is opened, if the header reports an
> encryption method, this is used to set the 'crypt_method_header'
> field on the BDRVQcow[2]State struct, and the 'encrypted' flag
> in the BDRVState struct.
>
> When doing I/O operations,
The qemu-io tool does not check if the image is encrypted so
historically would silently corrupt the sectors by writing
plain text data into them instead of cipher text. The earlier
commit turns this mistake into a fatal abort, so check for
encryption and prompt for key when required.
This enables
I realize that qcow[2] encryption is a feature we have deprecated
and will remove support for running it with the QEMU system
emulators in this cycle. We do still need to make sure it continues
to work for the sake of letting people run qemu-img convert to
retrieve their data though.
Some of the o
The qemu-img.c file has a read_password() method impl that is
used to prompt for passwords on the console, with impls for
POSIX and Windows. This will be needed by qemu-io.c too, so
move it into the QEMU osdep/oslib files where it can be shared
without code duplication
Signed-off-by: Daniel P. Ber
Add a simple test case for qemu-iotests that covers read/write
with encrypted qcow2 files.
Signed-off-by: Daniel P. Berrange
---
tests/qemu-iotests/131 | 69 ++
tests/qemu-iotests/131.out | 46 +++
tests/qemu-iotests/gro
When a qcow[2] file is opened, if the header reports an
encryption method, this is used to set the 'crypt_method_header'
field on the BDRVQcow[2]State struct, and the 'encrypted' flag
in the BDRVState struct.
When doing I/O operations, the 'crypt_method' field on the
BDRVQcow[2]State struct is che
The qemu_read_password() method looks for \r to terminate the
reading of the a password. This is what will be seen when
reading the password from a TTY. When scripting though, it is
useful to be able to send the password via a pipe, in which
case we must look for \n to terminate password input.
Si
On Tue 12 May 2015 05:10:56 PM CEST, Eric Blake wrote:
> POSIX says getopt() returns -1 on completion. While Linux happens
> to define EOF as -1, this definition is not required by POSIX, and
> there may be platforms where checking for EOF instead of -1 would
> lead to an infinite loop.
>
> Sign
POSIX says getopt() returns -1 on completion. While Linux happens
to define EOF as -1, this definition is not required by POSIX, and
there may be platforms where checking for EOF instead of -1 would
lead to an infinite loop.
Signed-off-by: Eric Blake
---
qemu-io-cmds.c | 16
qe
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf
> ---
> qemu-io-cmds.c | 71
> ++
> 1 file changed, 71 insertions(+)
>
> +
> +while ((c = getopt(argc, argv, "c:o:r")) != EOF) {
POSIX says getopt() returns -1 at
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf
> ---
> block.c | 42 +++---
> block/commit.c| 4 ++--
> include/block/block.h | 4 +++-
> 3 files changed, 44 insertions(+), 6 deletions(-)
>
> diff --git a/block.c
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> For bs->file, using references to existing BDSes has been possible for a
> while already. This patch enables the same for bs->backing_hd.
>
> Signed-off-by: Kevin Wolf
> ---
> block.c | 42 --
> blo
On 05/08/2015 11:21 AM, Kevin Wolf wrote:
> When reopening an image, the block layer already takes care to reopen
> bs->file as well with recalculated inherited flags. The same must happen
> for any other child (most notably missing before this patch: backing
> files).
>
> If bs->file (or any othe
I have used the following program to test
#define _GNU_SOURCE
#include
#include
#include
#include
#include
#include
int main(int argc, char *argv[])
{
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
void *buf;
int i = 0, align = atoi(argv[2]);
do {
buf =
The patch introduces new concept: minimal memory alignment for bounce
buffers. Original so called "optimal" value is actually minimal required
value for aligment. It should be used for validation that the IOVec
is properly aligned and bounce buffer is not required.
Though, from the performance poi
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10; i++)
write(fd, buf, 4096);
performs 5% better if buf is aligned to 4096 bytes.
The difference is quite reliable.
On the other hand we do not want at the moment to enforce
On 12/05/15 17:26, Kevin Wolf wrote:
Am 12.05.2015 um 16:20 hat Denis V. Lunev geschrieben:
On 12/05/15 17:08, Kevin Wolf wrote:
Am 12.05.2015 um 15:41 hat Denis V. Lunev geschrieben:
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10
Am 12.05.2015 um 16:20 hat Denis V. Lunev geschrieben:
> On 12/05/15 17:08, Kevin Wolf wrote:
> >Am 12.05.2015 um 15:41 hat Denis V. Lunev geschrieben:
> >>The following sequence
> >> int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
> >> for (i = 0; i < 10; i++)
> >>
Am 11.05.2015 um 17:45 hat Max Reitz geschrieben:
> On 08.05.2015 19:21, Kevin Wolf wrote:
> >This allows iterating over all children of a given BDS, not only
> >including bs->file and bs->backing_hd, but also driver-specific
> >ones like VMDK extents or Quorum children.
> >
> >Signed-off-by: Kevin
On 12/05/15 17:08, Kevin Wolf wrote:
Am 12.05.2015 um 15:41 hat Denis V. Lunev geschrieben:
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10; i++)
write(fd, buf, 4096);
performs 5% better if buf is aligned to 4096 byt
Am 12.05.2015 um 15:41 hat Denis V. Lunev geschrieben:
> The following sequence
> int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
> for (i = 0; i < 10; i++)
> write(fd, buf, 4096);
> performs 5% better if buf is aligned to 4096 bytes.
>
> The difference is quite
I have used the following program to test
#define _GNU_SOURCE
#include
#include
#include
#include
#include
#include
int main(int argc, char *argv[])
{
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
void *buf;
int i = 0, align = atoi(argv[2]);
do {
buf =
The patch introduces new concept: minimal memory alignment for bounce
buffers. Original so called "optimal" value is actually minimal required
value for aligment. It should be used for validation that the IOVec
is properly aligned and bounce buffer is not required.
Though, from the performance poi
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10; i++)
write(fd, buf, 4096);
performs 5% better if buf is aligned to 4096 bytes.
The difference is quite reliable.
On the other hand we do not want at the moment to enforce
Am 11.05.2015 um 17:20 hat Max Reitz geschrieben:
> On 08.05.2015 19:21, Kevin Wolf wrote:
> >Instead of letting every caller of bdrv_open() determine the right flags
> >for its child node manually and pass them to the function, pass the
> >parent node and the role of the newly opened child (like b
On 12/05/15 16:08, Kevin Wolf wrote:
Am 12.05.2015 um 12:36 hat Denis V. Lunev geschrieben:
On 12/05/15 13:27, Kevin Wolf wrote:
Am 12.05.2015 um 07:47 hat Denis V. Lunev geschrieben:
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10
Am 12.05.2015 um 12:36 hat Denis V. Lunev geschrieben:
> On 12/05/15 13:27, Kevin Wolf wrote:
> >Am 12.05.2015 um 07:47 hat Denis V. Lunev geschrieben:
> >>The following sequence
> >> int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
> >> for (i = 0; i < 10; i++)
> >>
On Tue, May 12, 2015 at 02:09:31PM +0800, Fam Zheng wrote:
> +static int coroutine_fn bdrv_co_do_zero_pwritev(BlockDriverState *bs,
> +int64_t offset,
> +unsigned int bytes,
> +
QEMU has options to configure the size of the L2 and refcount
caches for the qcow2 format. However, choosing the right sizes for
a particular disk image is not a straightforward operation since
the ratio between the cache size and the allocated disk space is
not obvious and depends on the size of t
Kevin applied all other patches from this series, so this is the only
one left.
Thanks to Max for all the feedback.
v2:
- Add copyright notice
- Document refcount_bits and its effects
v1: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg01891.html
Regards,
Berto
Alberto Garcia (1):
On 12/05/15 13:27, Kevin Wolf wrote:
Am 12.05.2015 um 07:47 hat Denis V. Lunev geschrieben:
The following sequence
int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
for (i = 0; i < 10; i++)
write(fd, buf, 4096);
performs 5% better if buf is aligned to 4096 byt
Am 12.05.2015 um 08:09 hat Fam Zheng geschrieben:
> Test zero write in byte range 512~1024 for 4k alignment.
>
> Signed-off-by: Fam Zheng
> Reviewed-by: Stefan Hajnoczi
Reviewed-by: Kevin Wolf
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
---
tests/qemu-iotests/041| 66 ++-
tests/qemu-iotests/iotests.py | 28 ++
2 files changed, 43 insertions(+), 51 deletions(-)
diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/
Am 12.05.2015 um 08:09 hat Fam Zheng geschrieben:
> For zero write, callers pass in NULL qiov (qemu-io "write -z" or
> scsi-disk "write same").
>
> Commit fc3959e466 fixed bdrv_co_write_zeroes which is the common case
> for this bug, but it still exists in bdrv_aio_write_zeroes. A simpler
> fix wo
Am 12.05.2015 um 08:09 hat Fam Zheng geschrieben:
> This reverts commit fc3959e4669a1c2149b91ccb05101cfc7ae1fc05.
>
> The core write code already handles the case, so remove this
> duplication.
>
> Because commit 61007b316 moved the touched code from block.c to
> block/io.c, the change is manuall
Only poll the specific type of event we are interested in, to avoid
stealing events that should be consumed by someone else.
Suggested-by: John Snow
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
---
tests/qemu-iotests/iotests.py | 9 ++---
1 file changed, 2 insertions(+), 7 deletions(-)
This checks that the discard on mirror source that effectively zeroes
data is also reflected by the data of target.
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
---
tests/qemu-iotests/131 | 59 ++
tests/qemu-iotests/131.out | 5
tests/qem
Using this function would always be wrong because a dirty bitmap must
have a specific owner that consumes the dirty bits and calls
bdrv_reset_dirty_bitmap().
Remove the unused function to avoid future misuse.
Reviewed-by: Eric Blake
Signed-off-by: Fam Zheng
Reviewed-by: John Snow
---
block.c
Unsetting dirty globally with discard is not very correct. The discard may zero
out sectors (depending on can_write_zeroes_with_unmap), we should replicate
this change to destinition side to make sure that the guest sees the same data.
Calling bdrv_reset_dirty also troubles mirror job because the
v3: Add John's rev-by in patches 3~6.
Rewrite patch 1: discard may not suffice, use write zeroes in that case.
v2: Fix typo and add Eric's rev-by in patch 3.
Add patch 1 to discard target in mirror job. (Paolo)
Add patch 6 to improve iotests.wait_ready. (John)
This fixes the mirror as
If guest discards a source cluster during mirror, we would want to
skip the read-write steps.
Source and target may have different "can_write_zeroes_with_unmap"
values and different discard granularity. It's important to replicate
the exact zeroing on the destination, so bdrv_aio_discard is only s
On 12/05/2015 12:27, Kevin Wolf wrote:
> I think it would make more sense to keep this specific to the raw-posix
> driver. After all, it's only the kernel page cache that we optimise
> here. Other backends probably don't take advantage of page alignment.
I don't think it makes sense to keep it r
On 12/05/2015 12:19, Denis V. Lunev wrote:
>
>
> hades /vol $ strace -f -e pwrite -e raw=write,pwrite qemu-io -n -c
> "write -P 0x11 0 64M" ./1.img
> Process 19326 attached
> [pid 19326] pwrite(0x6, 0x7fac07fff200, 0x400, 0x5) = 0x400
> < 1 GB Write from userspace
FWIW this is
Am 12.05.2015 um 07:47 hat Denis V. Lunev geschrieben:
> The patch introduces new concept: minimal memory alignment for bounce
> buffers. Original so called "optimal" value is actually minimal required
> value for aligment. It should be used for validation that the IOVec
> is properly aligned and b
Am 12.05.2015 um 07:47 hat Denis V. Lunev geschrieben:
> The following sequence
> int fd = open(argv[1], O_RDWR | O_CREAT | O_DIRECT, 0644);
> for (i = 0; i < 10; i++)
> write(fd, buf, 4096);
> performs 5% better if buf is aligned to 4096 bytes.
>
> The difference is quite
On 12/05/15 13:01, Stefan Hajnoczi wrote:
On Mon, May 11, 2015 at 07:47:41PM +0300, Denis V. Lunev wrote:
On 11/05/15 19:07, Denis V. Lunev wrote:
On 11/05/15 18:08, Stefan Hajnoczi wrote:
On Mon, May 04, 2015 at 04:42:22PM +0300, Denis V. Lunev wrote:
The difference is quite reliable and the
On Mon, May 11, 2015 at 07:47:41PM +0300, Denis V. Lunev wrote:
> On 11/05/15 19:07, Denis V. Lunev wrote:
> >On 11/05/15 18:08, Stefan Hajnoczi wrote:
> >>On Mon, May 04, 2015 at 04:42:22PM +0300, Denis V. Lunev wrote:
> >>>The difference is quite reliable and the same 5%.
> >>> qemu-io -n -c 'w
Am 11.05.2015 um 14:54 hat Alberto Garcia geschrieben:
> New version of the qcow2 cache patches:
>
> v3:
> - Removed a dead comment in patch #3
> - New document explaining how to configure the cache sizes
>
> v2: https://lists.nongnu.org/archive/html/qemu-devel/2015-05/msg00833.html
> - Don't do
75 matches
Mail list logo
