Re: [Qemu-block] RFC: Reducing the size of entries in the qcow2 L2 cache

Am 19.09.2017 um 17:07 hat Alberto Garcia geschrieben: > Hi everyone, > > over the past few weeks I have been testing the effects of reducing > the size of the entries in the qcow2 L2 cache. This was briefly > mentioned by Denis in the same thread where we discussed subcluster > allocation back

Re: [Qemu-block] [PATCH] throttle-groups: update tg->any_timer_armed[] on detach

On Tue 19 Sep 2017 05:50:25 PM CEST, Stefan Hajnoczi wrote: > Clear tg->any_timer_armed[] when throttling timers are destroy during > AioContext attach/detach. Failure to do so causes throttling to hang > because we believe the timer is already scheduled! > > The following was broken at least

Re: [Qemu-block] [PATCH v8 18/20] qcow2: Switch store_bitmap_data() to byte-based iteration

Am 19.09.2017 um 21:42 hat Eric Blake geschrieben: > However... > > >> -sbc = limit >> BDRV_SECTOR_BITS; > >> assert(DIV_ROUND_UP(bm_size, limit) == tb_size); > >> > >> -while ((sector = bdrv_dirty_iter_next(dbi) >> BDRV_SECTOR_BITS) >= 0) > >> { > >> -uint64_t cluster =

Re: [Qemu-block] [PATCH v3 03/20] file-posix: Switch to .bdrv_co_block_status()

On 09/20/2017 04:57 AM, Fam Zheng wrote: > On Thu, 09/14 09:40, Eric Blake wrote: >> We are gradually moving away from sector-based interfaces, towards >> byte-based. Update the file protocol driver accordingly. In mapping >> mode, note that the entire file is reported as allocated, so we can >>

Re: [Qemu-block] [PATCH 1/3] block: add bdrv_co_drain_end callback

On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote: > diff --git a/include/block/block_int.h b/include/block/block_int.h > index ba4c383393..ea1326e3c7 100644 > --- a/include/block/block_int.h > +++ b/include/block/block_int.h > @@ -359,6 +359,8 @@ struct BlockDriver { > */

Re: [Qemu-block] [PATCH v3 3/3] block/nbd-client: nbd_co_send_request: fix return code

On 09/20/2017 07:45 AM, Vladimir Sementsov-Ogievskiy wrote: > It's incorrect to return success rc >= 0 if we skip qio_channel_writev_all() > call due to s->quit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy > --- > block/nbd-client.c | 2 ++ > 1 file changed, 2

Re: [Qemu-block] [PATCH v3 0/3] nbd client refactoring and fixing

On 09/20/2017 07:45 AM, Vladimir Sementsov-Ogievskiy wrote: > v3: dropped all controversial things. I'll try to implement minimal > structured reply and block status over this small residue. > > Vladimir Sementsov-Ogievskiy (3): > block/nbd-client: refactor nbd_co_receive_reply >

Re: [Qemu-block] [PATCH 1/3] block: add bdrv_co_drain_end callback

On Wed, Sep 20, 2017 at 03:26:32PM +0100, Stefan Hajnoczi wrote: On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote: @@ -188,7 +194,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs) waited = BDRV_POLL_WHILE(bs, atomic_read(>in_flight) > 0); /* Ensure any

Re: [Qemu-block] [PATCH 1/2] block/block-backend.c: add blk_check_byte_request call to blk_pread/blk_pwrite

Am 20.09.2017 um 13:43 hat Manos Pitsidianakis geschrieben: > blk_check_byte_request() is called from the blk_co_pwritev/blk_co_preadv to > check if the request offset and request bytes parameters are valid for the > given Blockbackend. Let's do that in blk_pread/blk_pwrite too. > >

Re: [Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

On Wed 20 Sep 2017 01:39:02 PM CEST, Manos Pitsidianakis wrote: >>> void throttle_group_detach_aio_context(ThrottleGroupMember *tgm) >>> { >>> ThrottleTimers *tt = >throttle_timers; >>> +ThrottleGroup *tg = container_of(tgm->throttle_state, ThrottleGroup, >>> ts); >>> + >>> +

Re: [Qemu-block] ping Re: [PATCH v7 03/16] migration: split common postcopy out of ram postcopy

"Dr. David Alan Gilbert" wrote: > * Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: >> ping for 1-3 >> Can we merge them? > > I see all of them have R-b's; so lets try and put them in the next > migration merge. > > Quintela: Sound good? Yeap. Later, Juan.

Re: [Qemu-block] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin

On Wed, Sep 20, 2017 at 01:23:10PM +0300, Manos Pitsidianakis wrote: > Signed-off-by: Manos Pitsidianakis > --- > include/block/block_int.h | 2 +- > block/io.c| 4 ++-- > block/qed.c | 6 +++--- > 3 files changed, 6 insertions(+), 6

Re: [Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

On Wed, Sep 20, 2017 at 02:17:51PM +0200, Alberto Garcia wrote: > On Wed 20 Sep 2017 01:39:02 PM CEST, Manos Pitsidianakis wrote: > >>> void throttle_group_detach_aio_context(ThrottleGroupMember *tgm) > >>> { > >>> ThrottleTimers *tt = >throttle_timers; > >>> +ThrottleGroup *tg =

[Qemu-block] [PATCH v3 0/3] nbd client refactoring and fixing

v3: dropped all controversial things. I'll try to implement minimal structured reply and block status over this small residue. Vladimir Sementsov-Ogievskiy (3): block/nbd-client: refactor nbd_co_receive_reply block/nbd-client: simplify check in nbd_co_receive_reply block/nbd-client:

[Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

Clear tg->any_timer_armed[] when throttling timers are destroyed during AioContext attach/detach. Failure to do so causes throttling to hang because we believe the timer is already scheduled! The following was broken at least since QEMU 2.10.0 with -drive iops=100: $ dd if=/dev/zero

Re: [Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

On Wed, Sep 20, 2017 at 11:17:40AM +0100, Stefan Hajnoczi wrote: Clear tg->any_timer_armed[] when throttling timers are destroyed during AioContext attach/detach. Failure to do so causes throttling to hang because we believe the timer is already scheduled! The following was broken at least

[Qemu-block] [PATCH 1/2] block/block-backend.c: add blk_check_byte_request call to blk_pread/blk_pwrite

blk_check_byte_request() is called from the blk_co_pwritev/blk_co_preadv to check if the request offset and request bytes parameters are valid for the given Blockbackend. Let's do that in blk_pread/blk_pwrite too. Signed-off-by: Manos Pitsidianakis ---

[Qemu-block] [PATCH 1/3] block: add bdrv_co_drain_end callback

BlockDriverState has a bdrv_do_drain() callback but no equivalent for the end of the drain. The throttle driver (block/throttle.c) needs a way to mark the end of the drain in order to toggle io_limits_disabled correctly, thus bdrv_co_drain_end is needed. Signed-off-by: Manos Pitsidianakis

Re: [Qemu-block] RFC: Reducing the size of entries in the qcow2 L2 cache

On Wed 20 Sep 2017 09:06:20 AM CEST, Kevin Wolf wrote: >> |---+--+-+---+--| >> | Disk size | Cluster size | L2 cache| Standard QEMU | Patched QEMU | >> |---+--+-+---+--| >> | 16 GB

Re: [Qemu-block] [PATCH v2 5/7] block/nbd-client: nbd_co_send_request: return -EIO if s->quit was set in parallel

18.09.2017 19:01, Eric Blake wrote: On 09/18/2017 08:59 AM, Vladimir Sementsov-Ogievskiy wrote: It's incorrect to return success rc >= 0 if we skip qio_channel_writev_all() call due to s->quit. Does this need to cc: qemu-stable for 2.10.1 (or put another way, can we come up with some scenario

[Qemu-block] [PATCH 0/2] Truncate the tail of the image file in qcow2 shrinking

Now after shrinking the qcow2 image, at the end of the image file, there might be a tail that probably will never be used. Although it will not bring any tangible benefit, we can cut the tail if it is. Yes, it will not free up disk space, but if the blocks were be allocated sequentially and the

Re: [Qemu-block] [PATCH 1/2] qcow2: fix return error code in qcow2_truncate()

On 09/20/2017 08:58 AM, Pavel Butsykin wrote: > Signed-off-by: Pavel Butsykin > --- > block/qcow2.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Wow, bdrv_truncate() has had problems! You found fixes unrelated to my current struggles to fix the

Re: [Qemu-block] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks

On Wed, Sep 20, 2017 at 01:23:11PM +0300, Manos Pitsidianakis wrote: > Signed-off-by: Manos Pitsidianakis > --- > block/throttle.c | 18 ++ > 1 file changed, 18 insertions(+) Reviewed-by: Stefan Hajnoczi

[Qemu-block] [PATCH 0/2] remove blk_pread_unthrottled()

Cleanups for minor stuff I noticed while looking around blk_root_drained_* Manos Pitsidianakis (2): block/block-backend.c: add blk_check_byte_request call to blk_pread/blk_pwrite block/block-backend.c: remove blk_pread_unthrottled() include/sysemu/block-backend.h | 2 --

[Qemu-block] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin

Signed-off-by: Manos Pitsidianakis --- include/block/block_int.h | 2 +- block/io.c| 4 ++-- block/qed.c | 6 +++--- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/include/block/block_int.h b/include/block/block_int.h index

[Qemu-block] [PATCH 2/2] qcow2: truncate the tail of the image file after shrinking the image

Now after shrinking the image, at the end of the image file, there might be a tail that probably will never be used. So we can find the last used cluster and cut the tail. Signed-off-by: Pavel Butsykin --- block/qcow2-refcount.c | 21 + block/qcow2.c

Re: [Qemu-block] [PATCH 1/2] block/block-backend.c: add blk_check_byte_request call to blk_pread/blk_pwrite

On Wed, Sep 20, 2017 at 02:24:21PM +0200, Kevin Wolf wrote: Am 20.09.2017 um 13:43 hat Manos Pitsidianakis geschrieben: blk_check_byte_request() is called from the blk_co_pwritev/blk_co_preadv to check if the request offset and request bytes parameters are valid for the given Blockbackend.

[Qemu-block] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks

Signed-off-by: Manos Pitsidianakis --- block/throttle.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/block/throttle.c b/block/throttle.c index 5bca76300f..833175ac77 100644 --- a/block/throttle.c +++ b/block/throttle.c @@ -197,6 +197,21 @@ static

Re: [Qemu-block] [PATCH v3 03/20] file-posix: Switch to .bdrv_co_block_status()

On Thu, 09/14 09:40, Eric Blake wrote: > We are gradually moving away from sector-based interfaces, towards > byte-based. Update the file protocol driver accordingly. In mapping > mode, note that the entire file is reported as allocated, so we can > take a shortcut and skip lseek(). > >

Re: [Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

On Wed 20 Sep 2017 12:17:40 PM CEST, Stefan Hajnoczi wrote: > @@ -592,6 +592,17 @@ void > throttle_group_attach_aio_context(ThrottleGroupMember *tgm, > void throttle_group_detach_aio_context(ThrottleGroupMember *tgm) > { > ThrottleTimers *tt = >throttle_timers; > +ThrottleGroup *tg =

Re: [Qemu-block] [PATCH] block/throttle-groups.c: allocate RestartData on the heap

On Mon 18 Sep 2017 10:25:29 PM CEST, Manos Pitsidianakis wrote: > RestartData is the opaque data of the throttle_group_restart_queue_entry > coroutine. By being stack allocated, it isn't available anymore if > aio_co_enter schedules the coroutine with a bottom halve and runs after >

[Qemu-block] [PATCH 1/2] qcow2: fix return error code in qcow2_truncate()

Signed-off-by: Pavel Butsykin --- block/qcow2.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/qcow2.c b/block/qcow2.c index 2174a84d1f..8a4311d338 100644 --- a/block/qcow2.c +++ b/block/qcow2.c @@ -3166,7 +3166,7 @@ static int

Re: [Qemu-block] [PATCH] qemu-iotests/195: Change multiple nodes in one run

On 09/20/2017 07:15 AM, Kevin Wolf wrote: > This additional check for change-backing-file makes sure that actions on > one part of the graph don't interfere with later actions elsewhere. > > Suggested-by: Eric Blake > Signed-off-by: Kevin Wolf > --- >

Re: [Qemu-block] [PATCH 1/3] block: add bdrv_co_drain_end callback

On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote: > @@ -188,7 +194,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs) > waited = BDRV_POLL_WHILE(bs, atomic_read(>in_flight) > 0); > > /* Ensure any pending metadata writes are submitted to bs->file. */ > -

Re: [Qemu-block] [PATCH 0/6] block: Fix permissions after ro/rw reopen

Am 15.09.2017 um 12:10 hat Kevin Wolf geschrieben: > bdrv_reopen() can switch nodes between read-only and read-write modes. > This has implications for the required permissions on their child nodes. > For example, a qcow2 node requests write permissions on bs->file only if > it is writable itself.

Re: [Qemu-block] [Qemu-devel] [PATCH 15/18] block/mirror: Add active mirroring

On Tue, Sep 19, 2017 at 10:57:50AM +0100, Daniel P. Berrange wrote: > On Tue, Sep 19, 2017 at 10:44:16AM +0100, Stefan Hajnoczi wrote: > > On Mon, Sep 18, 2017 at 06:26:51PM +0200, Max Reitz wrote: > > > On 2017-09-18 12:06, Stefan Hajnoczi wrote: > > > > On Sat, Sep 16, 2017 at 03:58:01PM +0200,

Re: [Qemu-block] [PATCH 1/2] qcow2: fix return error code in qcow2_truncate()

On 09/20/2017 09:58 AM, Pavel Butsykin wrote: > Signed-off-by: Pavel Butsykin > --- > block/qcow2.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/block/qcow2.c b/block/qcow2.c > index 2174a84d1f..8a4311d338 100644 > --- a/block/qcow2.c >

Re: [Qemu-block] [Qemu-devel] [PATCH v9 05/20] dirty-bitmap: Avoid size query failure during truncate

On 09/20/2017 09:11 AM, Eric Blake wrote: > On 09/19/2017 09:10 PM, Fam Zheng wrote: > >>> >>> Do you suspect that almost certainly if bdrv_truncate() fails overall >>> that the image format driver will either unmount the image or become >>> read-only? > > Uggh - it feels like I've bitten off

Re: [Qemu-block] [PATCH 2/2] qcow2: truncate the tail of the image file after shrinking the image

On 09/20/2017 09:58 AM, Pavel Butsykin wrote: > Now after shrinking the image, at the end of the image file, there might be a > tail that probably will never be used. So we can find the last used cluster > and > cut the tail. > > Signed-off-by: Pavel Butsykin > --- >

Re: [Qemu-block] Block Migration and CPU throttling

Am 19.09.2017 um 16:41 schrieb Dr. David Alan Gilbert: > * Peter Lieven (p...@kamp.de) wrote: >> Am 19.09.2017 um 16:38 schrieb Dr. David Alan Gilbert: >>> * Peter Lieven (p...@kamp.de) wrote: Hi, I just noticed that CPU throttling and Block Migration don't work together very

Re: [Qemu-block] Block Migration and CPU throttling

Am 19.09.2017 um 16:41 schrieb Paolo Bonzini: > On 19/09/2017 15:36, Peter Lieven wrote: >> Hi, >> >> I just noticed that CPU throttling and Block Migration don't work >> together very well. >> During block migration the throttling heuristic detects that we >> obviously make no progress >> in ram

Re: [Qemu-block] [Qemu-devel] [PATCH 2/2] block/block-backend.c: remove blk_pread_unthrottled()

On 09/20/2017 07:43 AM, Manos Pitsidianakis wrote: > blk_pread_unthrottled was used to bypass I/O throttling on the BlockBackend in > the case of async I/O. This is not needed anymore and we can just call > blk_pread() directly. > > Signed-off-by: Manos Pitsidianakis >

[Qemu-block] [PATCH 2/2] ide: generic ide_data_write

Signed-off-by: John Snow --- hw/ide/core.c | 86 +-- hw/ide/trace-events | 3 +- include/hw/ide/internal.h | 1 + 3 files changed, 40 insertions(+), 50 deletions(-) diff --git a/hw/ide/core.c b/hw/ide/core.c index

[Qemu-block] [PATCH 0/2] IDE: combine portio r/w functions

Mark, here's a quick sketch for you. There are two things I don't like, but didn't care enough to fix: (1) Restricting nbytes to 2 or 4 means some extra boilerplate to quiet compilers who don't know it will only ever be 2 or 4 (2) the address value is all-but-ignored, it carries over from the

[Qemu-block] [PATCH v3 2/3] block/nbd-client: simplify check in nbd_co_receive_reply

If we are woken up from while() loop in nbd_read_reply_entry handles must be equal. If we are woken up from nbd_recv_coroutines_wake_all s->quit must be true, so we do not need checking handles equality. Signed-off-by: Vladimir Sementsov-Ogievskiy ---

[Qemu-block] [PATCH v3 3/3] block/nbd-client: nbd_co_send_request: fix return code

It's incorrect to return success rc >= 0 if we skip qio_channel_writev_all() call due to s->quit. Signed-off-by: Vladimir Sementsov-Ogievskiy --- block/nbd-client.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/nbd-client.c b/block/nbd-client.c index

[Qemu-block] [PATCH v3 1/3] block/nbd-client: refactor nbd_co_receive_reply

"NBDReply *reply" parameter of nbd_co_receive_reply is used only to pass return value for nbd_co_request (reply.error). Remove it and use function return value instead. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake ---

[Qemu-block] [PATCH] qemu-iotests/195: Change multiple nodes in one run

This additional check for change-backing-file makes sure that actions on one part of the graph don't interfere with later actions elsewhere. Suggested-by: Eric Blake Signed-off-by: Kevin Wolf --- tests/qemu-iotests/195 | 17

[Qemu-block] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks

This patch series renames bdrv_co_drain to bdrv_co_drain_begin and adds a new bdrv_co_drain_end callback to match bdrv_drained_begin/end and drained_begin/end of BdrvChild. This is needed because the throttle driver (block/throttle.c) needs a way to mark the end of the drain in order to toggle

[Qemu-block] [PATCH 2/2] block/block-backend.c: remove blk_pread_unthrottled()

blk_pread_unthrottled was used to bypass I/O throttling on the BlockBackend in the case of async I/O. This is not needed anymore and we can just call blk_pread() directly. Signed-off-by: Manos Pitsidianakis --- include/sysemu/block-backend.h | 2 -- block/block-backend.c

Re: [Qemu-block] [PATCH v2] throttle-groups: update tg->any_timer_armed[] on detach

On Wed, Sep 20, 2017 at 01:08:52PM +0200, Alberto Garcia wrote: On Wed 20 Sep 2017 12:17:40 PM CEST, Stefan Hajnoczi wrote: @@ -592,6 +592,17 @@ void throttle_group_attach_aio_context(ThrottleGroupMember *tgm, void throttle_group_detach_aio_context(ThrottleGroupMember *tgm) {

Re: [Qemu-block] [PATCH v3 2/3] block/nbd-client: simplify check in nbd_co_receive_reply

On 09/20/2017 07:45 AM, Vladimir Sementsov-Ogievskiy wrote: > If we are woken up from while() loop in nbd_read_reply_entry > handles must be equal. If we are woken up from > nbd_recv_coroutines_wake_all s->quit must be true, so we do > not need checking handles equality. > > Signed-off-by:

Re: [Qemu-block] [PATCH 2/2] block/block-backend.c: remove blk_pread_unthrottled()

Am 20.09.2017 um 13:43 hat Manos Pitsidianakis geschrieben: > blk_pread_unthrottled was used to bypass I/O throttling on the BlockBackend in > the case of async I/O. This is not needed anymore and we can just call > blk_pread() directly. > > Signed-off-by: Manos Pitsidianakis

Re: [Qemu-block] [Qemu-devel] [PATCH v9 05/20] dirty-bitmap: Avoid size query failure during truncate

On 09/19/2017 09:10 PM, Fam Zheng wrote: >> >> Do you suspect that almost certainly if bdrv_truncate() fails overall >> that the image format driver will either unmount the image or become >> read-only? Uggh - it feels like I've bitten off more than I can chew with this patch - I'm getting