Re: [Qemu-block] [PATCH 1/1] nbd: implement bdrv_get_info callback

2018-01-26 Thread Edgar Kaziakhmedov
PIng So, let me know if I need to make any changes in patch On 1/18/18 1:09 PM, Paolo Bonzini wrote: On 18/01/2018 12:51, Edgar Kaziakhmedov wrote: +static int nbd_get_info(BlockDriverState *bs, BlockDriverInfo *bdi) +{ +if (bs->supported_zero_flags & BDRV_REQ_MAY_UNMAP) { +

[Qemu-block] [PATCH v3 16/39] qcow2: Prepare l2_allocate() for adding L2 slice support

2018-01-26 Thread Alberto Garcia
Adding support for L2 slices to l2_allocate() needs (among other things) an extra loop that iterates over all slices of a new L2 table. Putting all changes in one patch would make it hard to read because all semantic changes would be mixed with pure indentation changes. To make things easier

[Qemu-block] [PATCH v3 15/39] qcow2: Update l2_load() to support L2 slices

2018-01-26 Thread Alberto Garcia
Each entry in the qcow2 L2 cache stores a full L2 table (which uses a complete cluster in the qcow2 image). A cluster is usually too large to be used efficiently as the size for a cache entry, so we want to decouple both values by allowing smaller cache entries. Therefore the qcow2 L2 cache will

[Qemu-block] [PATCH v3 02/39] qcow2: Add table size field to Qcow2Cache

2018-01-26 Thread Alberto Garcia
The table size in the qcow2 cache is currently equal to the cluster size. This doesn't allow us to use the cache memory efficiently, particularly with large cluster sizes, so we need to be able to have smaller cache tables that are independent from the cluster size. This patch adds a new field to

[Qemu-block] [PATCH v3 19/39] qcow2: Update get_cluster_table() to support L2 slices

2018-01-26 Thread Alberto Garcia
This patch updates get_cluster_table() to return L2 slices instead of full L2 tables. The code itself needs almost no changes, it only needs to call offset_to_l2_slice_index() instead of offset_to_l2_index(). This patch also renames all the relevant variables and the documentation.

[Qemu-block] [PATCH v3 37/39] iotests: Test valid values of l2-cache-entry-size

2018-01-26 Thread Alberto Garcia
The l2-cache-entry-size setting can only contain values that are powers of two between 512 and the cluster size. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/103 | 17 + tests/qemu-iotests/103.out | 3 +++ 2 files changed, 20 insertions(+) diff

[Qemu-block] [PATCH v3 39/39] iotests: Add l2-cache-entry-size to iotest 137

2018-01-26 Thread Alberto Garcia
This test tries reopening a qcow2 image with valid and invalid options. This patch adds l2-cache-entry-size to the set. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/137 | 5 + tests/qemu-iotests/137.out | 2 ++ 2 files changed, 7 insertions(+) diff --git

[Qemu-block] [PATCH v3 38/39] iotests: Test downgrading an image using a small L2 slice size

2018-01-26 Thread Alberto Garcia
expand_zero_clusters_in_l1() is used when downgrading qcow2 images from v3 to v2 (compat=0.10). This is one of the functions that needed more changes to support L2 slices, so this patch extends iotest 061 to test downgrading a qcow2 image using a smaller slice size. Signed-off-by: Alberto Garcia

[Qemu-block] [PATCH v3 08/39] qcow2: Remove BDS parameter from qcow2_cache_destroy()

2018-01-26 Thread Alberto Garcia
This function was never using the BlockDriverState parameter so it can be safely removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cache.c | 2 +- block/qcow2.c | 16 block/qcow2.h | 2 +- 3 files

[Qemu-block] [PATCH v3 11/39] qcow2: Remove BDS parameter from qcow2_cache_is_table_offset()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to pass it to qcow2_cache_get_table_addr(). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cache.c| 3 +--

[Qemu-block] [PATCH v3 04/39] qcow2: Remove BDS parameter from qcow2_cache_get_table_idx()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to get the cache table size (since it was equal to the cluster size). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake ---

[Qemu-block] [PATCH v3 35/39] qcow2: Rename l2_table in count_cow_clusters()

2018-01-26 Thread Alberto Garcia
This function doesn't need any changes to support L2 slices, but since it's now dealing with slices intead of full tables, the l2_table variable is renamed for clarity. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 4 ++--

[Qemu-block] [PATCH v3 17/39] qcow2: Update l2_allocate() to support L2 slices

2018-01-26 Thread Alberto Garcia
This patch updates l2_allocate() to support the qcow2 cache returning L2 slices instead of full L2 tables. The old code simply gets an L2 table from the cache and initializes it with zeroes or with the contents of an existing table. With a cache that returns slices instead of tables the idea

[Qemu-block] [PATCH v3 34/39] qcow2: Rename l2_table in count_contiguous_clusters_unallocated()

2018-01-26 Thread Alberto Garcia
This function doesn't need any changes to support L2 slices, but since it's now dealing with slices intead of full tables, the l2_table variable is renamed for clarity. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 6

[Qemu-block] [PATCH v3 28/39] qcow2: Read refcount before L2 table in expand_zero_clusters_in_l1()

2018-01-26 Thread Alberto Garcia
At the moment it doesn't really make a difference whether we call qcow2_get_refcount() before of after reading the L2 table, but if we want to support L2 slices we'll need to read the refcount first. This patch simply changes the order of those two operations to prepare for that. The patch with

[Qemu-block] [PATCH v3 27/39] qcow2: Update qcow2_update_snapshot_refcount() to support L2 slices

2018-01-26 Thread Alberto Garcia
qcow2_update_snapshot_refcount() increases the refcount of all clusters of a given snapshot. In order to do that it needs to load all its L2 tables and iterate over their entries. Since we'll be loading L2 slices instead of full tables we need to add an extra loop that iterates over all slices of

[Qemu-block] [PATCH v3 06/39] qcow2: Remove BDS parameter from qcow2_cache_entry_mark_dirty()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to pass it to qcow2_cache_get_table_idx(). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cache.c| 3 +--

[Qemu-block] [PATCH v3 12/39] qcow2: Add offset_to_l1_index()

2018-01-26 Thread Alberto Garcia
Similar to offset_to_l2_index(), this function returns the index in the L1 table for a given guest offset. This is only used in a couple of places and it's not a particularly complex calculation, but it makes the code a bit more readable. Although in the qcow2_get_cluster_offset() case the old

[Qemu-block] [PATCH v3 18/39] qcow2: Refactor get_cluster_table()

2018-01-26 Thread Alberto Garcia
After the previous patch we're now always using l2_load() in get_cluster_table() regardless of whether a new L2 table has to be allocated or not. This patch refactors that part of the code to use one single l2_load() call. Signed-off-by: Alberto Garcia ---

[Qemu-block] [PATCH v3 03/39] qcow2: Remove BDS parameter from qcow2_cache_get_table_addr()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to get the cache table size (since it was equal to the cluster size). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake ---

[Qemu-block] [PATCH v3 32/39] qcow2: Rename l2_table in qcow2_alloc_compressed_cluster_offset()

2018-01-26 Thread Alberto Garcia
This function doesn't need any changes to support L2 slices, but since it's now dealing with slices instead of full tables, the l2_table variable is renamed for clarity. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 16

[Qemu-block] [PATCH v3 01/39] qcow2: Fix documentation of get_cluster_table()

2018-01-26 Thread Alberto Garcia
This function has not been returning the offset of the L2 table since commit 3948d1d4876065160583e79533bf604481063833 Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)

[Qemu-block] [PATCH v3 20/39] qcow2: Update qcow2_get_cluster_offset() to support L2 slices

2018-01-26 Thread Alberto Garcia
qcow2_get_cluster_offset() checks how many contiguous bytes are available at a given offset. The returned number of bytes is limited by the amount that can be addressed without having to load more than one L2 table. Since we'll be loading L2 slices instead of full tables this patch changes the

[Qemu-block] [PATCH v3 13/39] qcow2: Add l2_slice_size field to BDRVQcow2State

2018-01-26 Thread Alberto Garcia
The BDRVQcow2State structure contains an l2_size field, which stores the number of 64-bit entries in an L2 table. For efficiency reasons we want to be able to load slices instead of full L2 tables, so we need to know how many entries an L2 slice can hold. An L2 slice is the portion of an L2

[Qemu-block] [PATCH v3 10/39] qcow2: Remove BDS parameter from qcow2_cache_discard()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to pass it to qcow2_cache_get_table_idx() and qcow2_cache_table_release(). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake ---

[Qemu-block] [PATCH v3 31/39] qcow2: Update qcow2_truncate() to support L2 slices

2018-01-26 Thread Alberto Garcia
The qcow2_truncate() code is mostly independent from whether we're using L2 slices or full L2 tables, but in full and falloc preallocation modes new L2 tables are allocated using qcow2_alloc_cluster_link_l2(). Therefore the code needs to be modified to ensure that all nb_clusters that are

[Qemu-block] [PATCH v3 00/39] Allow configuring the qcow2 L2 cache entry size

2018-01-26 Thread Alberto Garcia
this is the new revision of the patch series to allow configuring the entry size of the qcow2 L2 cache. Follow this link for the full description from the first version: https://lists.gnu.org/archive/html/qemu-block/2017-10/msg00458.html And here are some numbers showing the performance

[Qemu-block] [PATCH v3 09/39] qcow2: Remove BDS parameter from qcow2_cache_clean_unused()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to pass it to qcow2_cache_table_release(). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cache.c | 2 +-

[Qemu-block] [PULL 4/8] iotest 147: add cases to test new @name parameter of nbd-server-add

2018-01-26 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Eric Blake Message-Id: <20180119135719.24745-4-vsement...@virtuozzo.com> Signed-off-by: Eric Blake ---

Re: [Qemu-block] [PATCH v3 19/39] qcow2: Update get_cluster_table() to support L2 slices

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > This patch updates get_cluster_table() to return L2 slices instead of > full L2 tables. > > The code itself needs almost no changes, it only needs to call > offset_to_l2_slice_index() instead of offset_to_l2_index(). This patch > also renames all

Re: [Qemu-block] [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-26 Thread John Snow
On 01/24/2018 02:16 PM, Eric Blake wrote: > On 01/24/2018 12:17 AM, Liang Li wrote: >> We found that when doing drive mirror to a low speed shared storage, >> if there was heavy BLK IO write workload in VM after the 'ready' event, >> drive mirror block job can't be canceled immediately, it would

Re: [Qemu-block] [PATCH v2 3/6] qapi: add nbd-server-remove

2018-01-26 Thread Dr. David Alan Gilbert
* Vladimir Sementsov-Ogievskiy (vsement...@virtuozzo.com) wrote: > 17.01.2018 19:03, Eric Blake wrote: > > On 01/17/2018 09:51 AM, Vladimir Sementsov-Ogievskiy wrote: > > > > > > > I have a script (for managing libvirt guest, but it can be adopted for > > > > > qemu or even used for qemu

[Qemu-block] [PULL 8/8] nbd: implement bdrv_get_info callback

2018-01-26 Thread Eric Blake
From: Edgar Kaziakhmedov Since mirror job supports efficient zero out target mechanism (see in mirror_dirty_init()), implement bdrv_get_info to make it work over NBD. Such improvement will allow using the largest chunk possible and will decrease the number of

[Qemu-block] [PULL 1/8] qapi: add name parameter to nbd-server-add

2018-01-26 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Allow user to specify name for new export, to not reuse internal node name and to not show it to clients. This also allows creating several exports per device. Signed-off-by: Vladimir Sementsov-Ogievskiy

[Qemu-block] [PATCH v3 33/39] qcow2: Rename l2_table in count_contiguous_clusters()

2018-01-26 Thread Alberto Garcia
This function doesn't need any changes to support L2 slices, but since it's now dealing with slices intead of full tables, the l2_table variable is renamed for clarity. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cluster.c | 8

Re: [Qemu-block] [PATCH v3 16/39] qcow2: Prepare l2_allocate() for adding L2 slice support

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > Adding support for L2 slices to l2_allocate() needs (among other > things) an extra loop that iterates over all slices of a new L2 table. > > Putting all changes in one patch would make it hard to read because > all semantic changes would be mixed

Re: [Qemu-block] [PATCH] hmp: Add nbd_server_remove to mirror QMP command

2018-01-26 Thread Dr. David Alan Gilbert
* Eric Blake (ebl...@redhat.com) wrote: > Since everything else about the nbd-server-* QMP commands is > accessible from HMP, we might as well make removing an export > available as well. For now, I went with a bool flag rather > than a mode string for choosing between safe (default) and > hard

[Qemu-block] [PULL 3/8] qapi: add nbd-server-remove

2018-01-26 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Add command for removing an export. It is needed for cases when we don't want to keep the export after the operation on it was completed. The other example is a temporary node, created with blockdev-add. If we want to delete it we

[Qemu-block] [PATCH v3 30/39] qcow2: Update expand_zero_clusters_in_l1() to support L2 slices

2018-01-26 Thread Alberto Garcia
expand_zero_clusters_in_l1() expands zero clusters as a necessary step to downgrade qcow2 images to a version that doesn't support metadata zero clusters. This function takes an L1 table (which may or may not be active) and iterates over all its L2 tables looking for zero clusters. Since we'll be

[Qemu-block] [PULL 6/8] iotest 205: new test for qmp nbd-server-remove

2018-01-26 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20180119135719.24745-6-vsement...@virtuozzo.com> [eblake: adjust to next available test number] Signed-off-by: Eric Blake ---

[Qemu-block] [PATCH v3 23/39] qcow2: Update handle_alloc() to support L2 slices

2018-01-26 Thread Alberto Garcia
handle_alloc() loads an L2 table and limits the number of checked clusters to the amount that fits inside that table. Since we'll be loading L2 slices instead of full tables we need to update that limit. Apart from that, this function doesn't need any additional changes, so this patch simply

[Qemu-block] [PATCH v3 22/39] qcow2: Update handle_copied() to support L2 slices

2018-01-26 Thread Alberto Garcia
handle_copied() loads an L2 table and limits the number of checked clusters to the amount that fits inside that table. Since we'll be loading L2 slices instead of full tables we need to update that limit. Apart from that, this function doesn't need any additional changes, so this patch simply

[Qemu-block] [PATCH v3 24/39] qcow2: Update discard_single_l2() to support L2 slices

2018-01-26 Thread Alberto Garcia
discard_single_l2() limits the number of clusters to be discarded to the amount that fits inside an L2 table. Since we'll be loading L2 slices instead of full tables we need to update that limit. Apart from that, this function doesn't need any additional changes, so this patch simply updates the

Re: [Qemu-block] [Qemu-devel] [PATCH] block/mirror: fix fail to cancel when VM has heavy BLK IO

2018-01-26 Thread Eric Blake
On 01/26/2018 12:46 AM, Liang Li wrote: > The current QMP command is: > > { 'command': 'block-job-cancel', 'data': { 'device': 'str', '*force': 'bool' > } } > > 'force' has other meaning which is not used by libvirt, for the change, there > are 3 options: > > a. Now that 'force' is not used by

[Qemu-block] [PATCH v3 05/39] qcow2: Remove BDS parameter from qcow2_cache_table_release()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to get the cache table size (since it was equal to the cluster size). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake ---

[Qemu-block] [PATCH v3 26/39] qcow2: Prepare qcow2_update_snapshot_refcount() for adding L2 slice support

2018-01-26 Thread Alberto Garcia
Adding support for L2 slices to qcow2_update_snapshot_refcount() needs (among other things) an extra loop that iterates over all slices of each L2 table. Putting all changes in one patch would make it hard to read because all semantic changes would be mixed with pure indentation changes. To make

Re: [Qemu-block] [PATCH v3 17/39] qcow2: Update l2_allocate() to support L2 slices

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > This patch updates l2_allocate() to support the qcow2 cache returning > L2 slices instead of full L2 tables. > > The old code simply gets an L2 table from the cache and initializes it > with zeroes or with the contents of an existing table. With a

[Qemu-block] [PATCH v3 36/39] qcow2: Allow configuring the L2 slice size

2018-01-26 Thread Alberto Garcia
Now that the code is ready to handle L2 slices we can finally add an option to allow configuring their size. An L2 slice is the portion of an L2 table that is read by the qcow2 cache. Until now the cache was always reading full L2 tables, and since the L2 table size is equal to the cluster size

[Qemu-block] [PATCH v3 07/39] qcow2: Remove BDS parameter from qcow2_cache_put()

2018-01-26 Thread Alberto Garcia
This function was only using the BlockDriverState parameter to pass it to qcow2_cache_get_table_idx(). This is no longer necessary so this parameter can be removed. Signed-off-by: Alberto Garcia Reviewed-by: Eric Blake --- block/qcow2-cache.c| 2 +-

[Qemu-block] [PULL 5/8] iotests: implement QemuIoInteractive class

2018-01-26 Thread Eric Blake
From: Vladimir Sementsov-Ogievskiy Implement QemuIoInteractive to test nbd-server-remove command when there are active connections. Signed-off-by: Vladimir Sementsov-Ogievskiy Message-Id: <20180119135719.24745-5-vsement...@virtuozzo.com>

[Qemu-block] [PATCH v3 21/39] qcow2: Update qcow2_alloc_cluster_link_l2() to support L2 slices

2018-01-26 Thread Alberto Garcia
There's a loop in this function that iterates over the L2 entries in a table, so now we need to assert that it remains within the limits of an L2 slice. Apart from that, this function doesn't need any additional changes, so this patch simply updates the variable name from l2_table to l2_slice.

[Qemu-block] [PATCH v3 14/39] qcow2: Add offset_to_l2_slice_index()

2018-01-26 Thread Alberto Garcia
Similar to offset_to_l2_index(), this function takes a guest offset and returns the index in the L2 slice that contains its L2 entry. An L2 slice has currently the same size as an L2 table (one cluster), so both functions return the same value for now. Signed-off-by: Alberto Garcia

Re: [Qemu-block] [PATCH v3 18/39] qcow2: Refactor get_cluster_table()

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > After the previous patch we're now always using l2_load() in > get_cluster_table() regardless of whether a new L2 table has to be > allocated or not. > > This patch refactors that part of the code to use one single l2_load() > call. > >

[Qemu-block] [PATCH] block: Simplify bdrv_can_write_zeroes_with_unmap()

2018-01-26 Thread Eric Blake
We don't need the can_write_zeroes_with_unmap field in BlockDriverInfo, because it is redundant information with supported_zero_flags & BDRV_REQ_MAY_UNMAP. Note that BlockDriverInfo and supported_zero_flags are both per-device settings, rather than global state about the driver as a whole, which

Re: [Qemu-block] [PATCH v3 27/39] qcow2: Update qcow2_update_snapshot_refcount() to support L2 slices

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > qcow2_update_snapshot_refcount() increases the refcount of all > clusters of a given snapshot. In order to do that it needs to load all > its L2 tables and iterate over their entries. Since we'll be loading > L2 slices instead of full tables we need

Re: [Qemu-block] [PATCH v3 26/39] qcow2: Prepare qcow2_update_snapshot_refcount() for adding L2 slice support

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > Adding support for L2 slices to qcow2_update_snapshot_refcount() needs > (among other things) an extra loop that iterates over all slices of > each L2 table. > > Putting all changes in one patch would make it hard to read because > all semantic

Re: [Qemu-block] [PATCH v3 29/39] qcow2: Prepare expand_zero_clusters_in_l1() for adding L2 slice support

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > Adding support for L2 slices to expand_zero_clusters_in_l1() needs > (among other things) an extra loop that iterates over all slices of > each L2 table. > > Putting all changes in one patch would make it hard to read because > all semantic changes

Re: [Qemu-block] [PATCH v3 28/39] qcow2: Read refcount before L2 table in expand_zero_clusters_in_l1()

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > At the moment it doesn't really make a difference whether we call > qcow2_get_refcount() before of after reading the L2 table, but if we > want to support L2 slices we'll need to read the refcount first. > > This patch simply changes the order of

Re: [Qemu-block] [PATCH v3 30/39] qcow2: Update expand_zero_clusters_in_l1() to support L2 slices

2018-01-26 Thread Eric Blake
On 01/26/2018 08:59 AM, Alberto Garcia wrote: > expand_zero_clusters_in_l1() expands zero clusters as a necessary step > to downgrade qcow2 images to a version that doesn't support metadata > zero clusters. This function takes an L1 table (which may or may not > be active) and iterates over all

[Qemu-block] [RFC v3 14/14] iotests: test manual job dismissal

2018-01-26 Thread John Snow
RFC: The error returned by a job creation command when that device already has a job attached has become misleading; "Someone should do something about that!" Signed-off-by: John Snow --- tests/qemu-iotests/056 | 241 +

[Qemu-block] [RFC v3 13/14] blockjobs: Expose manual property

2018-01-26 Thread John Snow
Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow --- blockdev.c| 14 ++

[Qemu-block] [RFC v3 08/14] blockjobs: add commit, abort, clean helpers

2018-01-26 Thread John Snow
The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow --- blockjob.c | 35 ++- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git

[Qemu-block] [RFC v3 11/14] blockjobs: add PENDING status and event

2018-01-26 Thread John Snow
For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting," we add a new "pending" event.

[Qemu-block] [RFC v3 09/14] blockjobs: add prepare callback

2018-01-26 Thread John Snow
Some jobs, upon finalization, may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. Thus, we allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have

[Qemu-block] [RFC v3 12/14] blockjobs: add block-job-finalize

2018-01-26 Thread John Snow
Signed-off-by: John Snow --- block/trace-events | 1 + blockdev.c | 14 ++ blockjob.c | 72 +++- include/block/blockjob.h | 17 qapi/block-core.json | 18 5

[Qemu-block] [RFC v3 05/14] blockjobs: add block_job_dismiss

2018-01-26 Thread John Snow
For jobs that have reached their terminal state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via block-job-dismiss. This gives management APIs the

[Qemu-block] [RFC v3 10/14] blockjobs: Add waiting event

2018-01-26 Thread John Snow
For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their

[Qemu-block] [RFC v3 07/14] blockjobs: ensure abort is called for cancelled jobs

2018-01-26 Thread John Snow
Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's implementation of .commit checks for

[Qemu-block] [RFC v3 00/14] blockjobs: add explicit job management

2018-01-26 Thread John Snow
For jobs that complete when a monitor isn't looking, there's no way to tell what the job's final return code was. We need to allow jobs to remain in the list until queried for reliable management. Furthermore, it's not viable to have graph changes when the monitor isn't looking either. We need at

[Qemu-block] [RFC v3 01/14] blockjobs: add manual property

2018-01-26 Thread John Snow
This property will be used to opt-in to the new BlockJobs workflow that allows a tighter, more explicit control over transitions from one runstate to another. Signed-off-by: John Snow --- block/backup.c | 20 ++-- block/commit.c | 2

[Qemu-block] [RFC v3 06/14] blockjobs: add CONCLUDED state

2018-01-26 Thread John Snow
add a new state "CONCLUDED" that identifies a job that has ceased all operations. The wording was chosen to avoid any phrasing that might imply success, error, or cancellation. The task has simply ceased all operation and can never again perform any work. ("finished", "done", and "completed"

[Qemu-block] [RFC v3 04/14] blockjobs: RFC add block_job_verb permission table

2018-01-26 Thread John Snow
Which commands are appropriate for jobs in which state is also somewhat burdensome to keep track of. Introduce a verbs table that, as of this RFC patch, does nothing but serve as a declaration of intent. (At the very least, it forced me to consider all of the possibilities.) If this idea seems

[Qemu-block] [RFC v3 03/14] blockjobs: add state transition table

2018-01-26 Thread John Snow
The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. undefined: May only transition to 'created'. created: May only

[Qemu-block] [RFC v3 02/14] blockjobs: Add status enum

2018-01-26 Thread John Snow
We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. To this end, add a new "status" field and add our existing states in a redundant manner alongside the bools they are replacing: UNDEFINED: Placeholder, default state. CREATED: replaces