Re: [Qemu-block] [PATCH v7 00/10] block: Protect nested event loop with bdrv_drained_begin and bdrv_drained_end

Am 23.10.2015 um 05:08 hat Fam Zheng geschrieben: > v7: Exclude bdrv_drain and bdrv_qed_drain patches, they'll follow the > bdrv_drain fix for bdrv_aio_flush. > Fix internal snapshot clean. > > v6: Add Kevin's rev-by in patches 1-3, 6-8, 10, 12. > Add Jeff's rev-by in patches 1, 2,

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

On 23.10.2015 15:26, Kevin Wolf wrote: > Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: >> Signed-off-by: Max Reitz >> --- >> blockdev.c | 49 + >> qapi/block-core.json | 23 +++ >>

Re: [Qemu-block] [PATCH v7 05/10] block: Introduce "drained begin/end" API

On Fri, Oct 23, 2015 at 11:08:09AM +0800, Fam Zheng wrote: > +void bdrv_drained_begin(BlockDriverState *bs) > +{ > +if (!bs->quiesce_counter++) { > +aio_disable_external(bdrv_get_aio_context(bs)); > +} > +bdrv_drain(bs); > +} > + > +void bdrv_drained_end(BlockDriverState *bs) >

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

Am 23.10.2015 um 17:25 hat Max Reitz geschrieben: > On 23.10.2015 16:45, Kevin Wolf wrote: > > Am 23.10.2015 um 16:26 hat Max Reitz geschrieben: > >> On 23.10.2015 15:26, Kevin Wolf wrote: > >>> Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz >

[Qemu-block] [PATCH v3 4/4] iotests: Add tests for the x-blockdev-del command

Signed-off-by: Alberto Garcia --- tests/qemu-iotests/139 | 408 + tests/qemu-iotests/139.out | 5 + tests/qemu-iotests/group | 1 + 3 files changed, 414 insertions(+) create mode 100644 tests/qemu-iotests/139 create mode

Re: [Qemu-block] [PATCH v3 09/21] block: Add average I/O queue depth to BlockDeviceTimedStats

On Fri 23 Oct 2015 03:31:38 PM CEST, Stefan Hajnoczi wrote: >> +uint64_t timed_average_sum(TimedAverage *ta, uint64_t *elapsed) >> +{ >> +TimedAverageWindow *w; >> +check_expirations(ta); >> +w = current_window(ta); >> +if (elapsed != NULL) { >> +

Re: [Qemu-block] [PATCH v7 30/39] blockdev: Add blockdev-remove-medium

Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz I assume that you'll fix the bdrv_states issues in blk_insert/remove_bs, so the blk_remove_bs() call in this patch will take care of it. If so, you can add: Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz > --- > blockdev.c | 49 + > qapi/block-core.json | 23 +++ > qmp-commands.hx | 39

Re: [Qemu-block] [PATCH v7 31/39] blockdev: Add blockdev-insert-medium

On 23.10.2015 15:39, Kevin Wolf wrote: > Am 21.10.2015 um 15:47 hat Max Reitz geschrieben: >> On 21.10.2015 13:49, Alberto Garcia wrote: >>> On Mon 19 Oct 2015 05:53:37 PM CEST, Max Reitz wrote: And a helper function for that, which directly takes a pointer to the BDS to be inserted

Re: [Qemu-block] [PATCH v3 09/21] block: Add average I/O queue depth to BlockDeviceTimedStats

On Thu, Oct 22, 2015 at 11:11:19AM +0300, Alberto Garcia wrote: > +/* Get the sum of all accounted values > + * @ta: the TimedAverage structure > + * @elapsed: if non-NULL, the elapsed time (in ns) will be stored here > + * @ret: the sum of all accounted values > + */ > +uint64_t

Re: [Qemu-block] [PATCH v7 31/39] blockdev: Add blockdev-insert-medium

Am 21.10.2015 um 15:47 hat Max Reitz geschrieben: > On 21.10.2015 13:49, Alberto Garcia wrote: > > On Mon 19 Oct 2015 05:53:37 PM CEST, Max Reitz wrote: > >> And a helper function for that, which directly takes a pointer to the > >> BDS to be inserted instead of its node-name (which will be used

Re: [Qemu-block] [PATCH v3 08/21] block: Compute minimum, maximum and average I/O latencies

On Thu, Oct 22, 2015 at 11:11:18AM +0300, Alberto Garcia wrote: > +struct BlockAcctTimedStats { > +TimedAverage latency[BLOCK_MAX_IOTYPE]; > +unsigned interval_length; /* in seconds */ would be nice here so the units are clear. Or even interval_length_secs.

Re: [Qemu-block] [PATCH v7 29/39] blockdev: Add blockdev-close-tray

Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v3 00/21] Extended I/O accounting

On Thu, Oct 22, 2015 at 11:11:10AM +0300, Alberto Garcia wrote: > Here's a new version of the extended I/O accounting patches. > > This one is rebased on top of the current master, has a few minor > documentation fixes and drops the 'supports_stats' field completely. Looks good overall. I

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz > --- > blockdev.c | 49 + > qapi/block-core.json | 23 +++ > qmp-commands.hx | 39

[Qemu-block] [PATCH v3 1/4] mirror: block all operations on the target image during the job

There's nothing preventing the target image from being used by other operations during the 'drive-mirror' job, so we should block them all until the job is done. Signed-off-by: Alberto Garcia --- block/mirror.c | 4 1 file changed, 4 insertions(+) diff --git

[Qemu-block] [PATCH v3 2/4] block: Add blk_get_refcnt()

This function returns the reference count of a given BlockBackend. For convenience, it returns 0 if the BlockBackend pointer is NULL. Signed-off-by: Alberto Garcia Reviewed-by: Max Reitz --- block/block-backend.c | 5 +

[Qemu-block] [PATCH v3 3/4] block: Add 'x-blockdev-del' QMP command

This command is still experimental, hence the name. This is the companion to 'blockdev-add'. It allows deleting a BlockBackend with its associated BlockDriverState tree, or a BlockDriverState that is not attached to any backend. In either case, the command fails if the reference count is greater

[Qemu-block] [PATCH v3 0/4] Add 'x-blockdev-del' command

This version uses op blockers for the target image in the drive-mirror job, but the implementation of 'x-blockdev-del' remains the same. I copy the description from the previous series: The semantics of 'x-blockdev-del' try to mirror the semantics of 'blockdev-add' as I discussed with Kevin in

[Qemu-block] [PULL 14/37] block: Remove wr_highest_sector from BlockAcctStats

From: Max Reitz BlockAcctStats contains statistics about the data transferred from and to the device; wr_highest_sector does not fit in with the rest. Furthermore, those statistics are supposed to be specific for a certain device and not necessarily for a BDS (see the comment

[Qemu-block] [PULL 08/37] block/raw_bsd: Drop raw_is_inserted()

From: Max Reitz With the new automatically-recursive implementation of bdrv_is_inserted() checking by default whether all the children of a BDS are inserted, we can drop raw's own implementation. Signed-off-by: Max Reitz Reviewed-by: Eric Blake

[Qemu-block] [PULL 31/37] aio: introduce aio_{disable, enable}_external

From: Fam Zheng Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf --- aio-posix.c | 3 ++- aio-win32.c | 3 ++- include/block/aio.h | 38 ++ 3 files changed, 42 insertions(+), 2

[Qemu-block] [PULL 28/37] aio: Add "is_external" flag for event handlers

From: Fam Zheng All callers pass in false, and the real external ones will switch to true in coming patches. Signed-off-by: Fam Zheng Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 03/37] blockdev: Allow creation of BDS trees without BB

From: Max Reitz If the "id" field is missing from the options given to blockdev-add, just omit the BlockBackend and create the BlockDriverState tree alone. However, if "id" is missing, "node-name" must be specified; otherwise, the BDS tree would no longer be accessible. Many

[Qemu-block] [PULL 02/37] block: Set BDRV_O_INCOMING in bdrv_fill_options()

From: Max Reitz This flag should not be set for the root BDS only, but for any BDS that is being created while incoming migration is pending, so setting it is moved from blockdev_init() to bdrv_fill_options(). Signed-off-by: Max Reitz Reviewed-by: Eric

[Qemu-block] [PULL 21/37] block: Prepare remaining BB functions for NULL BDS

From: Max Reitz There are several BlockBackend functions which, in theory, cannot fail. This patch makes them cope with the BlockDriverState pointer being NULL by making them fall back to some default action like ignoring the value in setters and returning the default in

[Qemu-block] [PULL 35/37] block: Add "drained begin/end" for transactional blockdev-backup

From: Fam Zheng Similar to the previous patch, make sure that external events are not dispatched during transaction operations. Signed-off-by: Fam Zheng Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Signed-off-by: Kevin

[Qemu-block] [PULL 32/37] block: Introduce "drained begin/end" API

From: Fam Zheng The semantics is that after bdrv_drained_begin(bs), bs will not get new external requests until the matching bdrv_drained_end(bs). Signed-off-by: Fam Zheng Signed-off-by: Kevin Wolf --- block/io.c| 17

[Qemu-block] [PULL 36/37] block: Add "drained begin/end" for internal snapshot

From: Fam Zheng This ensures the atomicity of the transaction by avoiding processing of external requests such as those from ioeventfd. state->bs is assigned right after bdrv_drained_begin. Because it was used as the flag for deletion or not in abort, now we need a separate

[Qemu-block] [PULL 37/37] tests: Add test case for aio_disable_external

From: Fam Zheng Signed-off-by: Fam Zheng Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- tests/test-aio.c | 24 1 file changed, 24 insertions(+) diff --git a/tests/test-aio.c

[Qemu-block] [PULL 01/37] block: Remove host floppy support

From: Max Reitz It has been deprecated as of 2.3, so we can now remove it. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/raw-posix.c

[Qemu-block] [PULL 04/37] iotests: Only create BB if necessary

From: Max Reitz Tests 071 and 081 test giving references in blockdev-add. It is not necessary to create a BlockBackend here, so omit it. While at it, fix up some blockdev-add invocations in the vicinity (s/raw/$IMGFMT/ in 081, drop the format BDS for blkverify's raw child in

[Qemu-block] [PULL 20/37] block: Fail requests to empty BlockBackend

From: Max Reitz If there is no BlockDriverState in a BlockBackend or if the tray of the guest device is open, fail all requests (where that is possible) with -ENOMEDIUM. The reason the status of the guest device is taken into account is because once the guest device's tray is

[Qemu-block] [PULL 22/37] block: Add blk_insert_bs()

From: Max Reitz This function associates the given BlockDriverState with the given BlockBackend. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf --- block/block-backend.c | 11

[Qemu-block] [PULL 11/37] hw/usb-storage: Check whether BB is inserted

From: Max Reitz Only call bdrv_add_key() on the BlockDriverState if it is not NULL. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Alberto Garcia Reviewed-by: Kevin Wolf

[Qemu-block] [PULL 27/37] throttle: Remove throttle_group_lock/unlock()

From: Alberto Garcia The group throttling code was always meant to handle its locking internally. However, bdrv_swap() was touching the ThrottleGroup structure directly and therefore needed an API for that. Now that bdrv_swap() no longer exists there's no need for the

[Qemu-block] [PULL 29/37] nbd: Mark fd handlers client type as "external"

From: Fam Zheng So we could distinguish it from internal used fds, thus avoid handling unwanted events in nested aio polls. Signed-off-by: Fam Zheng Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 07/37] block: Make bdrv_is_inserted() recursive

From: Max Reitz If bdrv_is_inserted() is called on the top level BDS, it should make sure all nodes in the BDS tree are actually inserted. Signed-off-by: Max Reitz Reviewed-by: Eric Blake Reviewed-by: Kevin Wolf

[Qemu-block] [PULL 15/37] block: Move BlockAcctStats into BlockBackend

From: Max Reitz As the comment above bdrv_get_stats() says, BlockAcctStats is something which belongs to the device instead of each BlockDriverState. This patch therefore moves it into the BlockBackend. Signed-off-by: Max Reitz Reviewed-by: Eric Blake

[Qemu-block] [PULL 26/37] blockdev: Allow more options for BB-less BDS tree

From: Max Reitz Most of the options which blockdev_init() parses for both the BlockBackend and the root BDS are valid for just the root BDS as well (e.g. read-only). This patch allows specifying these options even if not creating a BlockBackend. Signed-off-by: Max Reitz

[Qemu-block] [PULL 19/37] block: Make some BB functions fall back to BBRS

From: Max Reitz If there is no BDS tree attached to a BlockBackend, functions that can do so should fall back to the BlockBackendRootState structure. Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf Signed-off-by: Kevin Wolf

[Qemu-block] [PULL 33/37] block: Add "drained begin/end" for transactional external snapshot

From: Fam Zheng This ensures the atomicity of the transaction by avoiding processing of external requests such as those from ioeventfd. Signed-off-by: Fam Zheng Reviewed-by: Jeff Cody Reviewed-by: Kevin Wolf Signed-off-by:

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

On 23.10.2015 16:45, Kevin Wolf wrote: > Am 23.10.2015 um 16:26 hat Max Reitz geschrieben: >> On 23.10.2015 15:26, Kevin Wolf wrote: >>> Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: Signed-off-by: Max Reitz --- blockdev.c | 49

Re: [Qemu-block] [PATCH v7 31/39] blockdev: Add blockdev-insert-medium

On 23.10.2015 15:42, Kevin Wolf wrote: > Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: >> And a helper function for that, which directly takes a pointer to the >> BDS to be inserted instead of its node-name (which will be used for >> implementing 'change' using blockdev-insert-medium). >> >>

Re: [Qemu-block] [PATCH v7 35/39] qmp: Introduce blockdev-change-medium

On 23.10.2015 16:25, Kevin Wolf wrote: > Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: >> Introduce a new QMP command 'blockdev-change-medium' which is intended >> to replace the 'change' command for block devices. The existing function >> qmp_change_blockdev() is accordingly renamed to >>

Re: [Qemu-block] [PATCH 1/4] ide/atapi: make PIO read requests async

Am 22.10.2015 um 18:17 schrieb Stefan Hajnoczi: > On Mon, Oct 12, 2015 at 02:27:22PM +0200, Peter Lieven wrote: >> @@ -129,9 +134,71 @@ static int cd_read_sector(IDEState *s, int lba, uint8_t >> *buf, int sector_size) >> ret = -EIO; >> break; >> } >> + >> +if (!ret) {

Re: [Qemu-block] [PATCH v7 05/10] block: Introduce "drained begin/end" API

On Fri, Oct 23, 2015 at 11:08:09AM +0800, Fam Zheng wrote: > +/** > + * bdrv_drained_begin: > + * > + * Begin a quiesced section for exclusive access to the BDS, by disabling > + * external request sources including NBD server and device model. Note that > + * this doesn't block timers or

Re: [Qemu-block] [PATCH v7 32/39] blockdev: Implement eject with basic operations

On 23.10.2015 15:54, Kevin Wolf wrote: > Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: >> Implement 'eject' by calling blockdev-open-tray and >> blockdev-remove-medium. >> >> Signed-off-by: Max Reitz >> --- >> blockdev.c | 11 +-- >> 1 file changed, 5

Re: [Qemu-block] [PATCH v7 28/39] blockdev: Add blockdev-open-tray

Am 23.10.2015 um 16:26 hat Max Reitz geschrieben: > On 23.10.2015 15:26, Kevin Wolf wrote: > > Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > >> Signed-off-by: Max Reitz > >> --- > >> blockdev.c | 49 > >> + > >>

Re: [Qemu-block] [PATCH 2/4] ide/atapi: blk_aio_readv may return NULL

Am 22.10.2015 um 18:20 schrieb Stefan Hajnoczi: > On Mon, Oct 12, 2015 at 02:27:23PM +0200, Peter Lieven wrote: >> Signed-off-by: Peter Lieven >> --- >> hw/ide/atapi.c | 4 >> 1 file changed, 4 insertions(+) >> >> diff --git a/hw/ide/atapi.c b/hw/ide/atapi.c >> index

Re: [Qemu-block] [PATCH v7 00/10] block: Protect nested event loop with bdrv_drained_begin and bdrv_drained_end

On Fri, Oct 23, 2015 at 11:08:04AM +0800, Fam Zheng wrote: > v7: Exclude bdrv_drain and bdrv_qed_drain patches, they'll follow the > bdrv_drain fix for bdrv_aio_flush. > Fix internal snapshot clean. > > v6: Add Kevin's rev-by in patches 1-3, 6-8, 10, 12. > Add Jeff's rev-by in patches

Re: [Qemu-block] [PATCH v7 35/39] qmp: Introduce blockdev-change-medium

Am 19.10.2015 um 17:53 hat Max Reitz geschrieben: > Introduce a new QMP command 'blockdev-change-medium' which is intended > to replace the 'change' command for block devices. The existing function > qmp_change_blockdev() is accordingly renamed to > qmp_blockdev_change_medium(). > >

[Qemu-block] [PATCH v10 10/14] block/backup: Rely on commit/abort for cleanup

Switch over to the new .commit/.abort handlers for cleaning up incremental bitmaps. [split up from a patch originally by Stefan and Fam. --js] Signed-off-by: Stefan Hajnoczi Signed-off-by: Fam Zheng Signed-off-by: John Snow

[Qemu-block] [PATCH v10 00/14] block: incremental backup transactions using BlockJobTxn

Welcome to V10! Where'd 8 and 9 go? Private off-list missives from Fam. Now you, I, and everyone on qemu-devel are staring at V10. What's new in V10? I replaced the per-action "transactional-cancel" parameter with a per-transaction paremeter named "err-cancel" which is implemented as an enum in

[Qemu-block] [PATCH v10 05/14] blockjob: Introduce reference count and fix reference to job->bs

From: Fam Zheng Add reference count to block job, meanwhile move the ownership of the reference to job->bs from the caller (which is released in two completion callbacks) to the block job itself. It is necessary for block_job_complete_sync to work, because block job shouldn't

[Qemu-block] [PATCH v10 01/14] qapi: Add transaction support to block-dirty-bitmap operations

This adds two qmp commands to transactions. block-dirty-bitmap-add allows you to create a bitmap simultaneously alongside a new full backup to accomplish a clean synchronization point. block-dirty-bitmap-clear allows you to reset a bitmap back to as-if it were new, which can also be used

[Qemu-block] [PATCH v10 03/14] block: rename BlkTransactionState and BdrvActionOps

These structures are misnomers, somewhat. (1) BlockTransactionState is not state for a transaction, but is rather state for a single transaction action. Rename it "BlkActionState" to be more accurate. (2) The BdrvActionOps describes operations for the BlkActionState, above. This name

[Qemu-block] [PATCH v10 06/14] blockjob: Add .commit and .abort block job actions

From: Fam Zheng Reviewed-by: Max Reitz Reviewed-by: John Snow Signed-off-by: Fam Zheng Signed-off-by: John Snow --- include/block/blockjob.h | 20 1 file changed, 20 insertions(+)

[Qemu-block] [PATCH v10 02/14] iotests: add transactional incremental backup test

Test simple usage cases for using transactions to create and synchronize incremental backups. Signed-off-by: John Snow Reviewed-by: Max Reitz Reviewed-by: Stefan Hajnoczi Signed-off-by: Stefan Hajnoczi Reviewed-by:

[Qemu-block] [PATCH v10 04/14] backup: Extract dirty bitmap handling as a separate function

From: Fam Zheng This will be reused by the coming new transactional completion code. Signed-off-by: Fam Zheng Reviewed-by: John Snow Reviewed-by: Max Reitz Signed-off-by: John Snow --- block/backup.c |

[Qemu-block] [PATCH v10 11/14] block: Add BlockJobTxn support to backup_run

Allow a BlockJobTxn to be passed into backup_run, which will allow the job to join a transactional group if present. Propagate this new parameter outward into new QMP helper functions in blockdev.c to allow transaction commands to pass forward their BlockJobTxn object in a forthcoming patch.

[Qemu-block] [PATCH v10 14/14] tests: add BlockJobTxn unit test

From: Stefan Hajnoczi The BlockJobTxn unit test verifies that both single jobs and pairs of jobs behave as a transaction group. Either all jobs complete successfully or the group is cancelled. Signed-off-by: Stefan Hajnoczi Reviewed-by: Max Reitz

[Qemu-block] [PATCH v10 08/14] blockjob: Simplify block_job_finish_sync

From: Fam Zheng With job->completed and job->ret to replace BlockFinishData. Signed-off-by: Fam Zheng Reviewed-by: Max Reitz Reviewed-by: John Snow Signed-off-by: John Snow --- blockjob.c | 27

[Qemu-block] [PATCH v10 13/14] iotests: 124 - transactional failure test

Use a transaction to request an incremental backup across two drives. Coerce one of the jobs to fail, and then re-run the transaction. Verify that no bitmap data was lost due to the partial transaction failure. To support the 'err-cancel' QMP argument name it's necessary for transaction_action()