[Qemu-block] [PATCH for-2.12] commit/stream: Reset delay_ns

2018-04-10 Thread Kevin Wolf
Streaming and the commit block job only want to apply throttling when they actually copied data instead of skipping it, so they made the calculation of delay_ns conditional. However, delay_ns isn't reset when skipping some sectors, so instead of not waiting, the old delay is applied again.

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Richard W.M. Jones
On Tue, Apr 10, 2018 at 03:48:11PM +0200, Kevin Wolf wrote: > Am 10.04.2018 um 15:03 hat Nir Soffer geschrieben: > > On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones > > wrote: > > > > > We now have true zeroing support in oVirt imageio, thanks for that. > > > > > > However

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones wrote: > We now have true zeroing support in oVirt imageio, thanks for that. > > However a problem is that ‘qemu-img convert’ issues zero requests for > the whole disk before starting the transfer. It does this using 32 MB >

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 4:48 PM Kevin Wolf wrote: > Am 10.04.2018 um 15:03 hat Nir Soffer geschrieben: > > On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones > > wrote: > > > > > We now have true zeroing support in oVirt imageio, thanks for that. > > > > > >

Re: [Qemu-block] [Libguestfs] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 6:00 PM Eric Blake wrote: > On 04/10/2018 09:40 AM, Richard W.M. Jones wrote: > >> When the destination is a block device we cannot avoid zeroing since a > block > >> device may contain junk data (we usually get dirty empty images from our > >> local >

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Eric Blake
On 04/10/2018 09:07 AM, Nir Soffer wrote: > On Tue, Apr 10, 2018 at 4:48 PM Kevin Wolf wrote: > >> Am 10.04.2018 um 15:03 hat Nir Soffer geschrieben: >>> On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones >>> wrote: >>> We now have true zeroing support

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Richard W.M. Jones
On Tue, Apr 10, 2018 at 02:07:33PM +, Nir Soffer wrote: > This makes sense if the device is backed by a block device on oVirt side, > and the NBD support efficient zeroing. But in this case the device is backed > by an empty sparse file on NFS, and oVirt does not support yet efficient >

Re: [Qemu-block] [Libguestfs] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Eric Blake
On 04/10/2018 09:40 AM, Richard W.M. Jones wrote: >> When the destination is a block device we cannot avoid zeroing since a block >> device may contain junk data (we usually get dirty empty images from our >> local >> xtremio server). > > (Off topic for qemu-block but ...) We don't have enough

Re: [Qemu-block] [Qemu-devel] [PATCH] qemu-iotests: Remove _supported_fmt dmg

2018-04-10 Thread Eric Blake
On 04/10/2018 03:42 AM, Kevin Wolf wrote: > qemu-iotests doesn't support dmg, and the dmg block driver doesn't > support image creation. Two test cases declare dmg as supported, but > that's obviously wrong for both reasons. Remove the declaration. > > Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 1/3] iotests: fix wait_until_completed()

2018-04-10 Thread Eric Blake
From: Peter Xu If there are more than one events, wait_until_completed() might return the 2nd event even if the 1st event is JOB_COMPLETED, since the for loop will continue to run even if completed is set to True. It never happened before, but it can be triggered when OOB is

Re: [Qemu-block] [PATCH for-2.12 v2] qemu-iotests: update 185 output

2018-04-10 Thread Kevin Wolf
Am 10.04.2018 um 10:11 hat Stefan Hajnoczi geschrieben: > On Wed, Apr 04, 2018 at 06:16:12PM +0200, Max Reitz wrote: > > On 2018-04-04 17:01, Stefan Hajnoczi wrote: > > === Start mirror job and exit qemu === > > > > This seems to be independent of whether there is actually data on > > TEST_IMG

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Nir Soffer
On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones wrote: > On Tue, Apr 10, 2018 at 02:07:33PM +, Nir Soffer wrote: > > This makes sense if the device is backed by a block device on oVirt side, > > and the NBD support efficient zeroing. But in this case the device is >

[Qemu-block] [PULL 7/7] qemu-iotests: update 185 output

2018-04-10 Thread Kevin Wolf
From: Stefan Hajnoczi Commit 4486e89c219c0d1b9bd8dfa0b1dd5b0d51ff2268 ("vl: introduce vm_shutdown()") added a bdrv_drain_all() call. As a side-effect of the drain operation the block job iterates one more time than before. The 185 output no longer matches and the test is

[Qemu-block] [PATCH 0/2] Minor fixes about compressed clusters with OFLAG_COPIED

2018-04-10 Thread Alberto Garcia
Hi, while reviewing one previous patch about data corruption and compressed clusters we discussed that the documentation doesn't clarify that L2 entries for compressed clusters are not supposed to have the OFLAG_COPIED bit set. Here's a patch to update the documentation and another one to fix

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Richard W.M. Jones
On Tue, Apr 10, 2018 at 03:25:47PM +, Nir Soffer wrote: > On Tue, Apr 10, 2018 at 5:50 PM Richard W.M. Jones > wrote: > > > On Tue, Apr 10, 2018 at 02:07:33PM +, Nir Soffer wrote: > > > This makes sense if the device is backed by a block device on oVirt side, > > > and

Re: [Qemu-block] [Qemu-devel] [PATCH for-2.12] commit/stream: Reset delay_ns

2018-04-10 Thread Eric Blake
On 04/10/2018 03:58 AM, Kevin Wolf wrote: > Streaming and the commit block job only want to apply throttling when > they actually copied data instead of skipping it, so they made the > calculation of delay_ns conditional. However, delay_ns isn't reset when > skipping some sectors, so instead of

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Kevin Wolf
Am 10.04.2018 um 15:03 hat Nir Soffer geschrieben: > On Tue, Apr 10, 2018 at 1:44 PM Richard W.M. Jones > wrote: > > > We now have true zeroing support in oVirt imageio, thanks for that. > > > > However a problem is that ‘qemu-img convert’ issues zero requests for > > the

[Qemu-block] [PULL 0/7] Block layer patches for 2.12.0-rc3

2018-04-10 Thread Kevin Wolf
The following changes since commit df6378eb0e6cfd58a22a1c3ff8fa4a9039f1eaa8: Merge remote-tracking branch 'remotes/kraxel/tags/ui-20180410-pull-request' into staging (2018-04-10 14:04:27 +0100) are available in the git repository at: git://repo.or.cz/qemu/kevin.git tags/for-upstream

[Qemu-block] [PULL 1/7] iotests.py: support unsupported_fmts in main()

2018-04-10 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Kevin Wolf --- tests/qemu-iotests/iotests.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git

Re: [Qemu-block] v2v: -o rhv-upload: Long time spent zeroing the disk

2018-04-10 Thread Richard W.M. Jones
On Tue, Apr 10, 2018 at 09:52:40AM -0500, Eric Blake wrote: > Didn't Rich already try to do that? > > +def emulate_zero(h, count, offset): > +# qemu-img convert starts by trying to zero/trim the whole device. > +# Since we've just created a new disk it's safe to ignore these > +#

[Qemu-block] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED

2018-04-10 Thread Alberto Garcia
Compressed clusters are not supposed to have the COPIED bit set. "qemu-img check" detects that and prints an error message reporting the number of the affected host cluster. This doesn't make much sense because compressed clusters are not aligned to host clusters, so it would be better to report

Re: [Qemu-block] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED

2018-04-10 Thread Eric Blake
On 04/10/2018 11:05 AM, Alberto Garcia wrote: > Compressed clusters are not supposed to have the COPIED bit set. > "qemu-img check" detects that and prints an error message reporting > the number of the affected host cluster. This doesn't make much sense > because compressed clusters are not

[Qemu-block] [PULL 2/7] hw/block/pflash_cfi: fix off-by-one error

2018-04-10 Thread Kevin Wolf
From: Philippe Mathieu-Daudé ASAN reported: hw/block/pflash_cfi02.c:245:33: runtime error: index 82 out of bounds for type 'uint8_t [82]' Since the 'cfi_len' member is not used, remove it to keep the code safer. Cc: qemu-sta...@nongnu.org Reported-by: AddressSanitizer

[Qemu-block] [PULL 6/7] commit/stream: Reset delay_ns

2018-04-10 Thread Kevin Wolf
Streaming and the commit block job only want to apply throttling when they actually copied data instead of skipping it, so they made the calculation of delay_ns conditional. However, delay_ns isn't reset when skipping some sectors, so instead of not waiting, the old delay is applied again.

[Qemu-block] [PULL 5/7] qemu-iotests: Remove _supported_fmt dmg

2018-04-10 Thread Kevin Wolf
qemu-iotests doesn't support dmg, and the dmg block driver doesn't support image creation. Two test cases declare dmg as supported, but that's obviously wrong for both reasons. Remove the declaration. Signed-off-by: Kevin Wolf Reviewed-by: Eric Blake ---

[Qemu-block] [PULL 3/7] iotests.py: improve verify_image_format helper

2018-04-10 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy Support "generic" formats like in bash tests with their _supported_fmt generic The test, supporting "generic" formats will run if IMGFMT_GENERIC = true, which is default, except for bochs and cloop. However, you can use

[Qemu-block] [PULL 4/7] iotests: blacklist bochs and cloop for 205 and 208

2018-04-10 Thread Kevin Wolf
From: Vladimir Sementsov-Ogievskiy Blacklist these formats, as they don't support image creation, as they say: > ./qemu-img create -f bochs x 1m qemu-img: x: Format driver 'bochs' does not support image creation > ./qemu-img create -f cloop x 1m

[Qemu-block] [PATCH 2/2] specs/qcow2: Clarify that compressed clusters have the COPIED bit reset

2018-04-10 Thread Alberto Garcia
Compressed clusters are not supposed to have the COPIED bit set, but this is not made explicit in the specs, so let's document it. Signed-off-by: Alberto Garcia --- docs/interop/qcow2.txt | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git

Re: [Qemu-block] [PATCH 1/2] Fix error message about compressed clusters with OFLAG_COPIED

2018-04-10 Thread Alberto Garcia
On Tue 10 Apr 2018 06:18:28 PM CEST, Eric Blake wrote: > On 04/10/2018 11:05 AM, Alberto Garcia wrote: >> Compressed clusters are not supposed to have the COPIED bit set. >> "qemu-img check" detects that and prints an error message reporting >> the number of the affected host cluster. This doesn't

Re: [Qemu-block] [PATCH for-2.12 v2] qemu-iotests: update 185 output

2018-04-10 Thread Stefan Hajnoczi
On Wed, Apr 04, 2018 at 06:16:12PM +0200, Max Reitz wrote: > On 2018-04-04 17:01, Stefan Hajnoczi wrote: > === Start mirror job and exit qemu === > > This seems to be independent of whether there is actually data on > TEST_IMG (the commit source), so something doesn't seem quite right with > the

Re: [Qemu-block] [PATCH v2 0/2] iotests: blacklist bochs and cloop for 205 and 208

2018-04-10 Thread Kevin Wolf
Am 09.04.2018 um 13:44 hat Vladimir Sementsov-Ogievskiy geschrieben: > v2: move from unsupported_fmts to support "generic", like in bash tests. Thanks, applied to the block branch. Kevin

[Qemu-block] [PATCH] qemu-iotests: Remove _supported_fmt dmg

2018-04-10 Thread Kevin Wolf
qemu-iotests doesn't support dmg, and the dmg block driver doesn't support image creation. Two test cases declare dmg as supported, but that's obviously wrong for both reasons. Remove the declaration. Signed-off-by: Kevin Wolf --- tests/qemu-iotests/183 | 2 +-

Re: [Qemu-block] [PATCH for-2.12] commit/stream: Reset delay_ns

2018-04-10 Thread Stefan Hajnoczi
On Tue, Apr 10, 2018 at 10:58:09AM +0200, Kevin Wolf wrote: > Streaming and the commit block job only want to apply throttling when > they actually copied data instead of skipping it, so they made the > calculation of delay_ns conditional. However, delay_ns isn't reset when > skipping some