[Qemu-block] [PATCH 0/3] qemu-img raw preallocation

2017-02-16 Thread Nir Soffer
This series add missing tests for raw image preallocation, refine preallocation=full and improve documentation. Create on top of the commit 10ddfe7b6044 (qemu-img: Do not truncate before preallocation) from Kevin block branch. Nir Soffer (3): qemu-img: Add tests for raw image preallocation

[Qemu-block] [PATCH 2/3] qemu-img: Truncate before full preallocation

2017-02-16 Thread Nir Soffer
In commit 10ddfe7b6044 (qemu-img: Do not truncate before preallocation) we moved truncate to the PREALLOC_MODE_OFF branch to avoid slowdown in posix_fallocate(). However this change is not optimal when using PREALLOC_MODE_FULL, since knowing the final size from the beginning could allow the file

[Qemu-block] [PATCH 1/3] qemu-img: Add tests for raw image preallocation

2017-02-16 Thread Nir Soffer
Add tests for creating raw image with and without the preallocation option. Signed-off-by: Nir Soffer --- tests/qemu-iotests/175 | 61 ++ tests/qemu-iotests/175.out | 18 ++ tests/qemu-iotests/group | 1 + 3 files

[Qemu-block] [PATCH 3/3] qemu-img: Improve documentation for PREALLOC_MODE_FALLOC

2017-02-16 Thread Nir Soffer
Now that we are truncating the file in both PREALLOC_MODE_FULL and PREALLOC_MODE_OFF, not truncating in PREALLOC_MODE_FALLOC looks odd. Add a comment explaining why we do not truncate in this case. Signed-off-by: Nir Soffer --- block/file-posix.c | 7 ++- 1 file changed, 6

[Qemu-block] [PATCH] iotests: Fix another race in 030

2017-02-16 Thread John Snow
We can't rely on a non-paused job to be present and running for us. Assume that if the job is not present that it completed already. Signed-off-by: John Snow --- tests/qemu-iotests/030 | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git

Re: [Qemu-block] [PATCH 12/17] migration: add postcopy migration of dirty bitmaps

2017-02-16 Thread Fam Zheng
On Mon, 02/13 12:54, Vladimir Sementsov-Ogievskiy wrote: > Postcopy migration of dirty bitmaps. Only named dirty bitmaps, > associated with root nodes and non-root named nodes are migrated. > > If destination qemu is already containing a dirty bitmap with the same name > as a migrated bitmap (for

Re: [Qemu-block] [PATCH v15 13/25] qcow2: add .bdrv_store_persistent_dirty_bitmaps()

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Realize block bitmap storing interface, to allow qcow2 images store > persistent bitmaps. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Max Reitz > Reviewed-by:

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Jaze Lee
2017-02-16 22:14 GMT+08:00 Jason Dillaman : > On Thu, Feb 16, 2017 at 4:00 AM, wrote: >> From: tianqing >> >> Rbd can do readv and writev directly, so wo do not need to transform >> iov to buf or vice versa any more. >> >>

Re: [Qemu-block] [PATCH v4 14/18] qcow2: add iotests to cover LUKS encryption support

2017-02-16 Thread Alberto Garcia
On Fri 10 Feb 2017 06:09:06 PM CET, Daniel P. Berrange wrote: > This extends the 087 iotest to cover LUKS encryption when doing > blockdev-add. > > Two further tests are added to validate read/write of LUKS > encrypted images with a single file and with a backing file. > > Reviewed-by: Max Reitz

Re: [Qemu-block] [PATCH v15 25/25] qcow2-bitmap: improve check_constraints_on_bitmap

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Add detailed error messages. > > Signed-off-by: Vladimir Sementsov-Ogievskiy Why not merge this patch into the one that originally introduced the function? Kevin

Re: [Qemu-block] [PATCH 10/18] block/dirty-bitmap: add bdrv_load_dirty_bitmap

2017-02-16 Thread Denis V. Lunev
On 02/03/2017 06:47 PM, Vladimir Sementsov-Ogievskiy wrote: > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/dirty-bitmap.c | 53 > > include/block/block_int.h| 4 > include/block/dirty-bitmap.h

Re: [Qemu-block] [PATCH v4 13/18] qcow2: add support for LUKS encryption format

2017-02-16 Thread Alberto Garcia
On Fri 10 Feb 2017 06:09:05 PM CET, Daniel P. Berrange wrote: > @@ -990,12 +1123,6 @@ static int qcow2_open(BlockDriverState *bs, QDict > *options, int flags, > s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1); > s->refcount_max += s->refcount_max - 1; > > -if

Re: [Qemu-block] [Qemu-devel] [PATCH 15/17] iotests: add default node-name

2017-02-16 Thread Fam Zheng
On Mon, 02/13 12:54, Vladimir Sementsov-Ogievskiy wrote: > When testing migration, auto-generated by qemu node-names differs in > source and destination qemu and migration fails. After this patch, > auto-generated by iotest nodenames will be the same. What should be done in libvirt to make sure

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Jaze Lee
No yet. I just test on one qemu-kvm vm. It works fine. The performance may need more time. Any one can test on this patch if you do fast 2017-02-16 20:07 GMT+08:00 Tiger Hu : > Tianqing, > > Do we have any performance data for this patch? Thanks. > > Tiger >> 在

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Jason Dillaman
On Thu, Feb 16, 2017 at 4:00 AM, wrote: > From: tianqing > > Rbd can do readv and writev directly, so wo do not need to transform > iov to buf or vice versa any more. > > Signed-off-by: tianqing > --- > block/rbd.c | 49

Re: [Qemu-block] [PATCH v15 21/25] qcow2-bitmap: refcounts

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Calculate refcounts for qcow2 bitmaps. It is needed for qcow2's qemu-img > check implementation. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > Reviewed-by: Max Reitz >

Re: [Qemu-block] [PATCH v15 09/25] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-16 Thread Kevin Wolf
Sorry, this was sent too early. Next attempt... Am 16.02.2017 um 12:45 hat Kevin Wolf geschrieben: > Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > > Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They > > are loaded when the image is opened and become

Re: [Qemu-block] [PATCH v15 08/25] block: introduce auto-loading bitmaps

2017-02-16 Thread Kevin Wolf
Am 16.02.2017 um 12:25 hat Kevin Wolf geschrieben: > Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > > Auto loading bitmaps are bitmaps stored in the disk image, which should > > be loaded when the image is opened and become BdrvDirtyBitmaps for the > > corresponding drive.

[Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread jazeltq
From: tianqing Rbd can do readv and writev directly, so wo do not need to transform iov to buf or vice versa any more. Signed-off-by: tianqing --- block/rbd.c | 49 ++--- 1 file changed, 42

Re: [Qemu-block] [PATCH v15 07/25] qcow2: add bitmaps extension

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Add bitmap extension as specified in docs/specs/qcow2.txt. > For now, just mirror extension header into Qcow2 state and check > constraints. > > For now, disable image resize if it has bitmaps. It will be fixed later. > >

Re: [Qemu-block] [PATCH v15 08/25] block: introduce auto-loading bitmaps

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Auto loading bitmaps are bitmaps stored in the disk image, which should > be loaded when the image is opened and become BdrvDirtyBitmaps for the > corresponding drive. > > Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [RFC v4] RBD: Add support readv,writev for rbd

2017-02-16 Thread jazeltq
From: tianqing Rbd can do readv and writev directly, so wo do not need to transform iov to buf or vice versa any more. Signed-off-by: tianqing --- block/rbd.c | 47 +-- 1 file changed, 41

Re: [Qemu-block] [PATCH v15 09/25] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-16 Thread Kevin Wolf
Am 15.02.2017 um 11:10 hat Vladimir Sementsov-Ogievskiy geschrieben: > Auto loading bitmaps are bitmaps in Qcow2, with the AUTO flag set. They > are loaded when the image is opened and become BdrvDirtyBitmaps for the > corresponding drive. > > Extra data in bitmaps is not supported for now. > >

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Jason Dillaman
On Thu, Feb 16, 2017 at 10:13 AM, Alexandre DERUMIER wrote: > Hi, I would like to bench it with small 4k read/write. > > On the ceph side,do we need this PR ? : > https://github.com/ceph/ceph/pull/13447 Yes, that is the correct PR for the client-side librbd changes. You

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Tiger Hu
Tianqing, Do we have any performance data for this patch? Thanks. Tiger > 在 2017年2月16日,下午5:00,jaze...@gmail.com 写道: > > From: tianqing > > Rbd can do readv and writev directly, so wo do not need to transform > iov to buf or vice versa any more. > > Signed-off-by:

Re: [Qemu-block] [PATCH 1/6] coroutine-lock: make CoMutex thread-safe

2017-02-16 Thread Stefan Hajnoczi
On Mon, Feb 13, 2017 at 07:12:39PM +0100, Paolo Bonzini wrote: > diff --git a/tests/test-aio-multithread.c b/tests/test-aio-multithread.c > index 534807d..ada8c48 100644 > --- a/tests/test-aio-multithread.c > +++ b/tests/test-aio-multithread.c > @@ -196,6 +196,88 @@ static void

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Alexandre DERUMIER
>>No yet. I just test on one qemu-kvm vm. It works fine. >>The performance may need more time. >>Any one can test on this patch if you do fast Hi, I would like to bench it with small 4k read/write. On the ceph side,do we need this PR ? : https://github.com/ceph/ceph/pull/13447 -

Re: [Qemu-block] [RFC v5] RBD: Add support readv,writev for rbd

2017-02-16 Thread Jeff Cody
On Thu, Feb 16, 2017 at 05:00:02PM +0800, jaze...@gmail.com wrote: > From: tianqing > > Rbd can do readv and writev directly, so wo do not need to transform > iov to buf or vice versa any more. > > Signed-off-by: tianqing > --- > block/rbd.c

Re: [Qemu-block] [PATCH 2/6] coroutine-lock: add limited spinning to CoMutex

2017-02-16 Thread Stefan Hajnoczi
On Mon, Feb 13, 2017 at 07:12:40PM +0100, Paolo Bonzini wrote: > Running a very small critical section on pthread_mutex_t and CoMutex > shows that pthread_mutex_t is much faster because it doesn't actually > go to sleep. What happens is that the critical section is shorter > than the latency of

Re: [Qemu-block] [PATCH 0/6] Make CoMutex/CoQueue/CoRwlock thread-safe

2017-02-16 Thread Stefan Hajnoczi
On Mon, Feb 13, 2017 at 07:12:38PM +0100, Paolo Bonzini wrote: > This is yet another tiny bit of the multiqueue work, this time affecting > the synchronization infrastructure for coroutines. Currently, coroutines > synchronize between the main I/O thread and the dataplane iothread through > the

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Kevin Wolf
Am 16.02.2017 um 19:23 hat Nir Soffer geschrieben: > On Thu, Feb 16, 2017 at 7:52 PM, Kevin Wolf wrote: > > Am 03.02.2017 um 20:50 hat Nir Soffer geschrieben: > >> When using file system that does not support fallocate() (e.g. NFS < > >> 4.2), truncating the file only when

Re: [Qemu-block] [PATCH 11/18] nbd: BLOCK_STATUS for bitmap export: server part

2017-02-16 Thread Denis V. Lunev
On 02/03/2017 06:47 PM, Vladimir Sementsov-Ogievskiy wrote: > Only one meta context type is defined: qemu-bitmap:. > Maximum one query is allowed for NBD_OPT_{SET,LIST}_META_CONTEXT, > NBD_REP_ERR_TOO_BIG is returned otherwise. > > Signed-off-by: Vladimir Sementsov-Ogievskiy

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Nir Soffer
Ping On Fri, Feb 3, 2017 at 9:50 PM, Nir Soffer wrote: > When using file system that does not support fallocate() (e.g. NFS < > 4.2), truncating the file only when preallocation=OFF speeds up creating > raw file. > > Here is example run, tested on Fedora 24 machine, creating

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Kevin Wolf
Am 03.02.2017 um 20:50 hat Nir Soffer geschrieben: > When using file system that does not support fallocate() (e.g. NFS < > 4.2), truncating the file only when preallocation=OFF speeds up creating > raw file. > > Here is example run, tested on Fedora 24 machine, creating raw file on > NFS version

Re: [Qemu-block] [PATCH] qemu-img: Do not truncate before preallocation

2017-02-16 Thread Nir Soffer
On Thu, Feb 16, 2017 at 7:52 PM, Kevin Wolf wrote: > Am 03.02.2017 um 20:50 hat Nir Soffer geschrieben: >> When using file system that does not support fallocate() (e.g. NFS < >> 4.2), truncating the file only when preallocation=OFF speeds up creating >> raw file. >> >> Here is

Re: [Qemu-block] [PATCH v15 09/25] qcow2: add .bdrv_load_autoloading_dirty_bitmaps

2017-02-16 Thread John Snow
On 02/16/2017 07:47 AM, Kevin Wolf wrote: > Can we make the whole thing a little less confusing by entirely removing > sectors from the block dirty bitmap interfaces? I find it challenging > enough to deal with just bytes, qcow2 clusters and bitmap clusters. I'll take a look to see how easy

Re: [Qemu-block] [Qemu-devel] [RFC v5] RBD: Add support readv, writev for rbd

2017-02-16 Thread Eric Blake
On 02/16/2017 03:00 AM, jaze...@gmail.com wrote: > From: tianqing > > Rbd can do readv and writev directly, so wo do not need to transform > iov to buf or vice versa any more. In general, we prefer new revisions of a patch series to be sent as a new top-level thread,

Re: [Qemu-block] [PATCH v3 7/8] blockdev: Make orphaned -drive fatal

2017-02-16 Thread John Snow
On 02/15/2017 05:05 AM, Markus Armbruster wrote: > Block backends defined with "-drive if=T" with T other than "none" are > meant to be picked up by machine initialization code: a suitable > frontend gets created and wired up automatically. > > If machine initialization code doesn't comply, the

Re: [Qemu-block] [Qemu-devel] [PATCH] iotests: Fix another race in 030

2017-02-16 Thread Fam Zheng
On Thu, 02/16 17:00, John Snow wrote: > We can't rely on a non-paused job to be present and running for us. > Assume that if the job is not present that it completed already. > > Signed-off-by: John Snow > --- > tests/qemu-iotests/030 | 5 - > 1 file changed, 4