Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/6] block: Mirror discarded sectors

2015-05-07 Thread Fam Zheng
On Thu, 05/07 14:20, Stefan Hajnoczi wrote: > On Wed, May 06, 2015 at 12:52:02PM +0800, Fam Zheng wrote: > > v2: Fix typo and add Eric's rev-by in patch 3. > > Add patch 1 to discard target in mirror job. (Paolo) > > Add patch 6 to improve iotests.wait_ready. (John) > > > > This fixes the

Re: [Qemu-block] [Qemu-devel] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-07 Thread Wen Congyang
On 05/08/2015 02:32 AM, John Snow wrote: > > > On 05/06/2015 10:20 PM, Wen Congyang wrote: >> On 05/02/2015 12:47 AM, John Snow wrote: >>> >>> >>> On 04/03/2015 07:05 AM, Paolo Bonzini wrote: On 03/04/2015 12:01, Wen Congyang wrote: > Signed-off-by: Wen Congyang > Signed-o

Re: [Qemu-block] [Qemu-devel] [PATCH COLO v3 10/14] util/hbitmap: Add an API to reset all set bits in hbitmap

2015-05-07 Thread John Snow
On 05/06/2015 10:20 PM, Wen Congyang wrote: > On 05/02/2015 12:47 AM, John Snow wrote: >> >> >> On 04/03/2015 07:05 AM, Paolo Bonzini wrote: >>> >>> >>> On 03/04/2015 12:01, Wen Congyang wrote: Signed-off-by: Wen Congyang Signed-off-by: zhanghailiang Signed-off-by: Gonglei

Re: [Qemu-block] [Qemu-devel] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-07 Thread John Snow
On 05/07/2015 10:54 AM, Stefan Hajnoczi wrote: > On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: >> +static void block_dirty_bitmap_clear_prepare(BlkTransactionState >> *common, + Error >> **errp) +{ +BlockDirtyBitmapState *state = >> DO_

[Qemu-block] [PATCH v4] block/vdi: Use bdrv_flush after metadata updates

2015-05-07 Thread Zhe Qiu
In reference to b0ad5a45...078a458e, metadata writes to qcow2/cow/qcow/vpc/vmdk are all synced prior to succeeding writes. Only when write is successful that bdrv_flush is called. Signed-off-by: Zhe Qiu --- block/vdi.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/vdi.c b/block/

Re: [Qemu-block] [PATCH v3] block/vdi: Use bdrv_flush after metadata updates

2015-05-07 Thread Eric Blake
On 05/06/2015 10:04 PM, Zhe Qiu wrote: > From: phoeagon > > In reference to > b0ad5a455d7e5352d4c86ba945112011dbeadfb8~078a458e077d6b0db262c4b05fee51d01de2d1d2, > metadata writes to qcow2/cow/qcow/vpc/vmdk are all synced prior to > succeeding writes. > Please wrap commit comments to be under

Re: [Qemu-block] [PATCH 3/7] qcow2: use an LRU algorithm to replace entries from the L2 cache

2015-05-07 Thread Alberto Garcia
On Thu 07 May 2015 04:55:21 PM CEST, Eric Blake wrote: >> /* Give the table some hits for the start so that it won't be replaced >> * immediately. The number 32 is completely arbitrary. */ >> -c->entries[i].cache_hits = 32; >> c->entries[i].offset = offset; > > The comment is n

Re: [Qemu-block] [PATCH v3 01/10] qapi: Add transaction support to block-dirty-bitmap operations

2015-05-07 Thread Stefan Hajnoczi
On Wed, Apr 22, 2015 at 08:04:44PM -0400, John Snow wrote: > +static void block_dirty_bitmap_clear_prepare(BlkTransactionState *common, > + Error **errp) > +{ > +BlockDirtyBitmapState *state = DO_UPCAST(BlockDirtyBitmapState, > +

Re: [Qemu-block] [PATCH 3/7] qcow2: use an LRU algorithm to replace entries from the L2 cache

2015-05-07 Thread Eric Blake
On 05/06/2015 07:39 AM, Alberto Garcia wrote: > The current algorithm to evict entries from the cache gives always > preference to those in the lowest positions. As the size of the cache > increases, the chances of the later elements of being removed decrease > exponentially. > > In a scenario wit

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Paolo Bonzini
On 07/05/2015 16:34, Kevin Wolf wrote: > Am 07.05.2015 um 16:16 hat Paolo Bonzini geschrieben: >> >> >> On 07/05/2015 16:07, Kevin Wolf wrote: >>> This is not right for two reasons: The first is that this is >>> BlockBackend code >> >> I think it would take effect for the qemu-nbd case though. >

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Kevin Wolf
Am 07.05.2015 um 16:16 hat Paolo Bonzini geschrieben: > > > On 07/05/2015 16:07, Kevin Wolf wrote: > > This is not right for two reasons: The first is that this is > > BlockBackend code > > I think it would take effect for the qemu-nbd case though. Oh, you want to change the server code rather

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Paolo Bonzini
On 07/05/2015 16:07, Kevin Wolf wrote: > This is not right for two reasons: The first is that this is > BlockBackend code I think it would take effect for the qemu-nbd case though. > and it wouldn't even take effect for the qcow2 case > where we're writing past EOF only on the protocol layer. T

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Kevin Wolf
Am 07.05.2015 um 15:55 hat Paolo Bonzini geschrieben: > > > On 07/05/2015 15:20, Kevin Wolf wrote: > > > Does ENOSPC over LVM (dm-linear) work at all, and who generates the > > > ENOSPC there? > > > > The LVM use case is what oVirt uses, so I'm pretty sure that it works. > > I'm now sure who gene

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Paolo Bonzini
On 07/05/2015 15:20, Kevin Wolf wrote: > > Does ENOSPC over LVM (dm-linear) work at all, and who generates the > > ENOSPC there? > > The LVM use case is what oVirt uses, so I'm pretty sure that it works. > I'm now sure who generates the ENOSPC, but it's not qemu anyway. If I > had to guess, I'd s

Re: [Qemu-block] [Qemu-devel] [PATCH] nvme: support NVME_VOLATILE_WRITE_CACHE feature

2015-05-07 Thread Kevin Wolf
[ Cc: qemu-block ] Am 30.04.2015 um 16:03 hat Keith Busch geschrieben: > On Thu, 30 Apr 2015, Christoph Hellwig wrote: > >The SCSI emulation in the Linux NVMe driver really wants to know > >if a device has a volatile write cache. Given that qemu has moved > >away from a model where we report the

Re: [Qemu-block] [RFC PATCH 0/7] Fix transactional snapshot with virtio-blk dataplane

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 07:23:32PM +0800, Fam Zheng wrote: > Reported by Paolo. > > Unlike the iohandler in main loop, iothreads currently process the event > notifier used as virtio-blk ioeventfd in all nested aio_poll. This is > dangerous > without proper protection, because guest requests coul

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Kevin Wolf
Am 07.05.2015 um 14:47 hat Paolo Bonzini geschrieben: > > > On 07/05/2015 14:29, Kevin Wolf wrote: > > > No, it shouldn't indeed. > > > > > > Could alloc_clusters_noref do bdrv_truncate and return ENOSPC if it > > > fails? That's how for example qcow and vhdx work. vdi has the same > > > probl

Re: [Qemu-block] [PATCH v2 0/6] block: Mirror discarded sectors

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 12:52:02PM +0800, Fam Zheng wrote: > v2: Fix typo and add Eric's rev-by in patch 3. > Add patch 1 to discard target in mirror job. (Paolo) > Add patch 6 to improve iotests.wait_ready. (John) > > This fixes the mirror assert failure reported by wangxiaolong: > > htt

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Paolo Bonzini
On 07/05/2015 14:29, Kevin Wolf wrote: > > No, it shouldn't indeed. > > > > Could alloc_clusters_noref do bdrv_truncate and return ENOSPC if it > > fails? That's how for example qcow and vhdx work. vdi has the same > > problem. > > If you want NBD to return -ENOSPC for writes after EOF, chang

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Kevin Wolf
Am 07.05.2015 um 14:20 hat Paolo Bonzini geschrieben: > > > On 06/05/2015 19:23, Max Reitz wrote: > > The guest sees whatever has been written into reply->error, and that > > field hasn't been written by this function in that case. It has been > > written by nbd_receive_reply() in nbd.c, and that

Re: [Qemu-block] [Qemu-devel] [PATCH v2 0/3] block: Warn about usage of growing formats over non-growable protocols

2015-05-07 Thread Paolo Bonzini
On 06/05/2015 19:23, Max Reitz wrote: > The guest sees whatever has been written into reply->error, and that > field hasn't been written by this function in that case. It has been > written by nbd_receive_reply() in nbd.c, and that value comes directly > from the server. In case of qemu-nbd being

Re: [Qemu-block] [Qemu-devel] [PATCH 7/7] qcow2: style fixes in qcow2-cache.c

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:31PM +0300, Alberto Garcia wrote: > Fix pointer declaration to make it consistent with the rest of the > code. > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cache.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) Reviewed-by: Stefan Hajnoczi

Re: [Qemu-block] [Qemu-devel] [PATCH 6/7] qcow2: make qcow2_cache_put() a void function

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:30PM +0300, Alberto Garcia wrote: > This function never receives an invalid table pointer, so we can make > it void and remove all the error checking code. > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cache.c| 7 +-- > block/qcow2-cluster.c | 50

Re: [Qemu-block] [PATCH 5/7] qcow2: use a hash to look for entries in the L2 cache

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:29PM +0300, Alberto Garcia wrote: > The current cache algorithm traverses the array starting always from > the beginning, so the average number of comparisons needed to perform > a lookup is proportional to the size of the array. > > By using a hash of the offset as t

Re: [Qemu-block] [Qemu-devel] [PATCH 4/7] qcow2: remove qcow2_cache_find_entry_to_replace()

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:28PM +0300, Alberto Garcia wrote: > A cache miss means that the whole array was traversed and the entry > we were looking for was not found, so there's no need to traverse it > again in order to select an entry to replace. > > Signed-off-by: Alberto Garcia > --- > b

Re: [Qemu-block] [PATCH 3/7] qcow2: use an LRU algorithm to replace entries from the L2 cache

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:27PM +0300, Alberto Garcia wrote: > The current algorithm to evict entries from the cache gives always > preference to those in the lowest positions. As the size of the cache > increases, the chances of the later elements of being removed decrease > exponentially. > >

Re: [Qemu-block] [PATCH 2/7] qcow2: simplify qcow2_cache_put() and qcow2_cache_entry_mark_dirty()

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:26PM +0300, Alberto Garcia wrote: > Since all tables are now stored together, it is possible to obtain > the position of a particular table directly from its address, so the > operation becomes O(1). > > Signed-off-by: Alberto Garcia > --- > block/qcow2-cache.c | 32

Re: [Qemu-block] [Qemu-devel] [PATCH 1/7] qcow2: use one single memory block for the L2/refcount cache tables

2015-05-07 Thread Stefan Hajnoczi
On Wed, May 06, 2015 at 04:39:25PM +0300, Alberto Garcia wrote: > The qcow2 L2/refcount cache contains one separate table for each cache > entry. Doing one allocation per table adds unnecessary overhead and it > also requires us to store the address of each table separately. > > Since the size of

Re: [Qemu-block] [Qemu-devel] [PATCH v3] block/vdi: Use bdrv_flush after metadata updates

2015-05-07 Thread Stefan Hajnoczi
On Thu, May 07, 2015 at 12:04:56PM +0800, Zhe Qiu wrote: > From: phoeagon > > In reference to > b0ad5a455d7e5352d4c86ba945112011dbeadfb8~078a458e077d6b0db262c4b05fee51d01de2d1d2, > metadata writes to qcow2/cow/qcow/vpc/vmdk are all synced prior to > succeeding writes. > > Only when write is s

Re: [Qemu-block] [PATCH 5/6] qcow2: use a hash to look for entries in the L2 cache

2015-05-07 Thread Alberto Garcia
On Wed 06 May 2015 06:42:30 PM CEST, Stefan Hajnoczi wrote: >> By using a hash of the offset as the starting point, lookups are >> faster and independent from the array size. >> >> The hash is computed using the cluster number of the table, multiplied >> by 4 to make it perform better when there