[PATCH v11 08/13] copy-on-read: add support for BDRV_REQ_PREFETCH to COR-filter

2020-10-12 Thread Andrey Shinkevich via
Add support for the BDRV_REQ_PREFETCH flag to the supported_write_flags of the COR-filter. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index dfbd6ad..b136895 100644 ---

[PATCH v11 09/13] copy-on-read: skip non-guest reads if no copy needed

2020-10-12 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 13

[PATCH v11 10/13] stream: skip filters when writing backing file name to QCOW2 header

2020-10-12 Thread Andrey Shinkevich via
Avoid writing a filter JSON-name to QCOW2 image when the backing file is changed after the block stream job. Signed-off-by: Andrey Shinkevich --- block/stream.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..51462bd

[PATCH v11 07/13] block: include supported_read_flags into BDS structure

2020-10-12 Thread Andrey Shinkevich via
Add the new member supported_read_flags to BlockDriverState structure. It will control the BDRV_REQ_PREFETCH flag set for copy-on-read operations. Signed-off-by: Andrey Shinkevich --- include/block/block_int.h | 4 1 file changed, 4 insertions(+) diff --git a/include/block/block_int.h

[PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-12 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests

[PATCH v11 12/13] stream: remove unused backing-file name parameter

2020-10-12 Thread Andrey Shinkevich via
The 'backing-file' argument is not used by the block-stream job. It designates a backing file name to set in QCOW2 image header after the block-stream job finished. A backing file name of the node above base is used instead. Signed-off-by: Andrey Shinkevich --- block/stream.c| 6

[PATCH v11 11/13] stream: mark backing-file argument as deprecated

2020-10-12 Thread Andrey Shinkevich via
Whereas the block-stream job starts using a backing file name of the base node overlay after the block-stream job completes, mark the QMP 'backing-file' argument as deprecated. Signed-off-by: Andrey Shinkevich --- docs/system/deprecated.rst | 6 ++ 1 file changed, 6 insertions(+) diff

[PATCH v11 01/13] copy-on-read: Support preadv/pwritev_part functions

2020-10-12 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v11 00/13] Apply COR-filter to the block-stream permanently

2020-10-12 Thread Andrey Shinkevich via
The iotest case test_stream_parallel still does not pass after the COR-filter is inserted into the backing chain. As the test case may not be initialized, it does not make a sense and was removed again. v11: 04: Base node overlay is used instead of base. 05: Base node overlay is used instead

[PATCH v11 06/13] block: modify the comment for BDRV_REQ_PREFETCH flag

2020-10-12 Thread Andrey Shinkevich via
Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- include/block/block.h | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git

[PATCH v11 05/13] copy-on-read: limit COR operations to base in COR driver

2020-10-12 Thread Andrey Shinkevich via
Limit COR operations by the base node in the backing chain when the overlay base node name is given. It will be useful for a block stream job when the COR-filter is applied. The overlay base node is passed as the base itself may change due to concurrent commit jobs on the same backing chain.

[PATCH v11 03/13] qapi: add filter-node-name to block-stream

2020-10-12 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH v11 02/13] copy-on-read: add filter append/drop functions

2020-10-12 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 88

[PATCH v11 04/13] copy-on-read: pass overlay base node name to COR driver

2020-10-12 Thread Andrey Shinkevich via
We are going to use the COR-filter for a block-stream job. To limit COR operations by the base node in the backing chain during stream job, pass the name of overlay base node to the copy-on-read driver as base node itself may change due to possible concurrent jobs. The rest of the functionality

[PATCH v8 2/7] copy-on-read: add filter append/drop functions

2020-08-28 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 104

[PATCH v8 1/7] copy-on-read: Support preadv/pwritev_part functions

2020-08-28 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v8 0/7] Apply COR-filter to the block-stream permanently

2020-08-28 Thread Andrey Shinkevich via
Note: this series is based on the another one "block: Deal with filters" by Max Reitz that could be found in the branches: https://git.xanclic.moe/XanClic/qemu child-access-functions-v6 https://github.com/XanClic/qemu child-access-functions-v6 v8: 03: qapi - version changed to

[PATCH v8 6/7] block-stream: freeze link to base node during stream job

2020-08-28 Thread Andrey Shinkevich via
To keep the base node unchanged during the block-stream operation, freeze it as the other part of the backing chain with the intermediate nodes related to the job. This patch revers the change made with the commit c624b015bf as the correct base file name and its format have to be written down to

[PATCH v8 3/7] qapi: add filter-node-name to block-stream

2020-08-28 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH v8 4/7] copy-on-read: pass base file name to COR driver

2020-08-28 Thread Andrey Shinkevich via
To limit the guest's COR operations by the base node in the backing chain during stream job, pass the base file name to the copy-on-read driver. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 41

[PATCH v8 5/7] copy-on-read: limit guest writes to base in COR driver

2020-08-28 Thread Andrey Shinkevich via
Limit the guest's COR operations by the base node in the backing chain during a stream job. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 49 + 1 file changed, 49 insertions(+) diff --git a/block/copy-on-read.c

[PATCH v8 7/7] block: apply COR-filter to block-stream jobs

2020-08-28 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests

[PATCH v10 7/9] stream: skip filters when writing backing file name to QCOW2 header

2020-09-29 Thread Andrey Shinkevich via
Avoid writing a filter JSON-name to QCOW2 image when the backing file is changed after the block stream job. Signed-off-by: Andrey Shinkevich --- block/stream.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..b0719e9

[PATCH v10 9/9] block: apply COR-filter to block-stream jobs

2020-09-29 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests

[PATCH v10 5/9] copy-on-read: limit guest COR activity to base in COR driver

2020-09-29 Thread Andrey Shinkevich via
Limit the guest's COR operations by the base node in the backing chain when the base node name is given. It will be useful for a block stream job when the COR-filter is applied. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 38 -- 1 file

[PATCH v10 8/9] block: remove unused backing-file name parameter

2020-09-29 Thread Andrey Shinkevich via
The block stream QMP parameter backing-file is in use no more. It designates a backing file name to set in QCOW2 image header after the block stream job finished. The base file name is used instead. Signed-off-by: Andrey Shinkevich --- block/monitor/block-hmp-cmds.c | 2 +- block/stream.c

[PATCH v10 2/9] copy-on-read: add filter append/drop functions

2020-09-29 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 84 block/copy-on-read.h | 35

[PATCH v10 6/9] copy-on-read: skip non-guest reads if no copy needed

2020-09-29 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 14

[PATCH v10 4/9] copy-on-read: pass base node name to COR driver

2020-09-29 Thread Andrey Shinkevich via
To limit the guest's COR operations by the base node in the backing chain during stream job, pass the base node name to the copy-on-read driver. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 13

[PATCH v10 3/9] qapi: add filter-node-name to block-stream

2020-09-29 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH v10 0/9] Apply COR-filter to the block-stream permanently

2020-09-29 Thread Andrey Shinkevich via
Despite the patch "freeze link to base node..." has been removed from the series in the current version 9, the iotest case test_stream_parallel does not pass after the COR-filter is inserted into the backing chain. As the test case may not be initialized, it does not make a sense and was removed

[PATCH v10 1/9] copy-on-read: Support preadv/pwritev_part functions

2020-09-29 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v9 3/9] qapi: add filter-node-name to block-stream

2020-09-28 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH v9 5/9] copy-on-read: limit guest COR activity to base in COR driver

2020-09-28 Thread Andrey Shinkevich via
Limit the guest's COR operations by the base node in the backing chain when the base node name is given. It will be useful for a block stream job when the COR-filter is applied. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 38 -- 1 file

[PATCH v9 6/9] copy-on-read: skip non-guest reads if no copy needed

2020-09-28 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 14

[PATCH v9 4/9] copy-on-read: pass base node name to COR driver

2020-09-28 Thread Andrey Shinkevich via
To limit the guest's COR operations by the base node in the backing chain during stream job, pass the base node name to the copy-on-read driver. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 13

[PATCH v9 7/9] stream: skip filters when writing backing file name to QCOW2 header

2020-09-28 Thread Andrey Shinkevich via
Avoid writing a filter JSON-name to QCOW2 image when the backing file is changed after the block stream job. Signed-off-by: Andrey Shinkevich --- block/stream.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..b0719e9

[PATCH v9 0/9] Apply COR-filter to the block-stream permanently

2020-09-28 Thread Andrey Shinkevich via
Despite the patch "freeze link to base node..." has been removed from the series in the current version 9, the iotest case test_stream_parallel does not pass after the COR-filter is inserted into the backing chain. As the test case may not be initialized, it does not make a sense and was removed

[PATCH v9 9/9] block: apply COR-filter to block-stream jobs

2020-09-28 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests

[PATCH v9 8/9] block: remove unused backing-file name parameter

2020-09-28 Thread Andrey Shinkevich via
The block stream QMP parameter backing-file is in use no more. It designates a backing file name to set in QCOW2 image header after the block stream job finished. The base file name is used instead. Signed-off-by: Andrey Shinkevich --- block/monitor/block-hmp-cmds.c | 2 +- block/stream.c

[PATCH v9 1/9] copy-on-read: Support preadv/pwritev_part functions

2020-09-28 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v9 2/9] copy-on-read: add filter append/drop functions

2020-09-28 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 84 1 file changed, 84

[PATCH v3 0/5] Increase amount of data for monitor to read

2020-11-27 Thread Andrey Shinkevich via
The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html https://patchew.org/QEMU/20190610105906.28524-1-dplotni...@virtuozzo.com/# Message-ID: <31dd78ba-bd64-2ed6-3c8f-eed4e904d...@virtuozzo.com> and v2: Message-Id:

[PATCH v3 4/5] iotests: 129 don't check backup "busy"

2020-11-27 Thread Andrey Shinkevich via
From: Vladimir Sementsov-Ogievskiy Busy is racy, job has it's "pause-points" when it's not busy. Drop this check. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/129 | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/qemu-iotests/129

[PATCH v3 3/5] monitor: let QMP monitor track JSON message content

2020-11-27 Thread Andrey Shinkevich via
We are going to allow the QMP monitor reading data from input channel more than one byte at once to increase the performance. With the OOB compatibility disabled, the monitor queues one QMP command at most. It was done for the backward compatibility as stated in the comment before pushing a

[PATCH v3 2/5] monitor: drain requests queue with 'channel closed' event

2020-11-27 Thread Andrey Shinkevich via
When CHR_EVENT_CLOSED comes, the QMP requests queue may still contain unprocessed commands. It can happen with QMP capability OOB enabled. Let the dispatcher complete handling requests rest in the monitor queue. Signed-off-by: Andrey Shinkevich --- monitor/qmp.c | 46

[PATCH v3 5/5] monitor: increase amount of data for monitor to read

2020-11-27 Thread Andrey Shinkevich via
QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse. This patch increases the amount of read data up to 4096 bytes that fits the buffer size on the channel level.

[PATCH v3 1/5] monitor: change function obsolete name in comments

2020-11-27 Thread Andrey Shinkevich via
The function name monitor_qmp_bh_dispatcher() has been changed to monitor_qmp_dispatcher_co() since the commit 9ce44e2c. Let's amend the comments. Signed-off-by: Andrey Shinkevich --- monitor/qmp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/monitor/qmp.c

[PATCH v2 2/2] monitor: increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse. This patch increases the amount of read data up to 4096 bytes that fits the buffer size on the channel level. A

[PATCH v2 1/2] iotests: add another bash sleep command to 247

2020-11-23 Thread Andrey Shinkevich via
This patch paves the way for the one that follows. The following patch makes the QMP monitor to read up to 4K from stdin at once. That results in running the bash 'sleep' command before the _qemu_proc_exec() starts in subshell. Another 'sleep' command with an unobtrusive 'query-status' plays as a

[PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from the input buffer so that incoming requests do not

[PATCH v13 01/10] copy-on-read: support preadv/pwritev_part functions

2020-12-02 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v13 02/10] block: add API function to insert a node

2020-12-02 Thread Andrey Shinkevich via
Provide API for insertion a node to backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block.c | 25 + include/block/block.h | 2 ++ 2 files changed, 27 insertions(+) diff --git

[PATCH v13 00/10] Apply COR-filter to the block-stream permanently

2020-12-02 Thread Andrey Shinkevich via
The previous version 12 was discussed in the email thread: Message-Id: <1603390423-980205-1-git-send-email-andrey.shinkev...@virtuozzo.com> v13: 02: The bdrv_remove_node() was dropped. 05: Three patches with fixes were merged into one. 06: Minor changes based on Vladimir's suggestions.

[PATCH v13 05/10] qapi: create BlockdevOptionsCor structure for COR driver

2020-12-02 Thread Andrey Shinkevich via
Create the BlockdevOptionsCor structure for COR driver specific options splitting it off form the BlockdevOptionsGenericFormat. The only option 'bottom' node in the structure denotes an image file that limits the COR operations in the backing chain. We are going to use the COR-filter for a

[PATCH v13 03/10] copy-on-read: add filter drop function

2020-12-02 Thread Andrey Shinkevich via
Provide API for the COR-filter removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. To insert the filter, the block generic layer function bdrv_insert_node() can be used. The new function bdrv_cor_filter_drop() may be considered as an

[PATCH v13 10/10] block: apply COR-filter to block-stream jobs

2020-12-02 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter applied to block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the test case

[PATCH v13 08/10] copy-on-read: skip non-guest reads if no copy needed

2020-12-02 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write operations in COR-driver. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Add the BDRV_REQ_PREFETCH flag to the supported_read_flags of the

[PATCH v13 04/10] qapi: add filter-node-name to block-stream

2020-12-02 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH v13 09/10] stream: skip filters when writing backing file name to QCOW2 header

2020-12-02 Thread Andrey Shinkevich via
Avoid writing a filter JSON file name and a filter format name to QCOW2 image when the backing file is being changed after the block stream job. It can occur due to a concurrent commit job on the same backing chain. A user is still able to assign the 'backing-file' parameter for a block-stream job

[PATCH v13 07/10] block: include supported_read_flags into BDS structure

2020-12-02 Thread Andrey Shinkevich via
Add the new member supported_read_flags to the BlockDriverState structure. It will control the flags set for copy-on-read operations. Make the block generic layer evaluate supported read flags before they go to a block driver. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey

[PATCH v13 06/10] iotests: add #310 to test bottom node in COR driver

2020-12-02 Thread Andrey Shinkevich via
The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/310 | 114

[PATCH v12 14/14] block: apply COR-filter to block-stream jobs

2020-10-22 Thread Andrey Shinkevich via
This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse. The COR-filter insertion incurs changes in the iotests

[PATCH v12 01/14] copy-on-read: support preadv/pwritev_part functions

2020-10-22 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions(+),

[PATCH v12 05/14] qapi: create BlockdevOptionsCor structure for COR driver

2020-10-22 Thread Andrey Shinkevich via
Create the BlockdevOptionsCor structure for COR driver specific options splitting it off form the BlockdevOptionsGenericFormat. The only option 'bottom' node in the structure denotes an image file that limits the COR operations in the backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey

[PATCH v12 11/14] copy-on-read: add support for read flags to COR-filter

2020-10-22 Thread Andrey Shinkevich via
Add the BDRV_REQ_COPY_ON_READ and BDRV_REQ_PREFETCH flags to the supported_read_flags of the COR-filter. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 8178a91..a2b180a 100644 ---

[PATCH v12 10/14] block: include supported_read_flags into BDS structure

2020-10-22 Thread Andrey Shinkevich via
Add the new member supported_read_flags to the BlockDriverState structure. It will control the flags set for copy-on-read operations. Make the block generic layer evaluate supported read flags before they go to a block driver. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey

[PATCH v12 12/14] copy-on-read: skip non-guest reads if no copy needed

2020-10-22 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write operations in COR-driver. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 12

[PATCH v12 07/14] copy-on-read: limit COR operations to bottom node

2020-10-22 Thread Andrey Shinkevich via
Limit COR operations to the bottom node (inclusively) in the backing chain when the bottom node name is given. It will be useful for a block stream job when the COR-filter is applied. The bottom node is passed as the base itself may change due to concurrent commit jobs on the same backing chain.

[PATCH v12 08/14] iotests: add #310 to test bottom node in COR driver

2020-10-22 Thread Andrey Shinkevich via
The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/310 | 109 + tests/qemu-iotests/310.out | 15 +++

[PATCH v12 02/14] block: add insert/remove node functions

2020-10-22 Thread Andrey Shinkevich via
Provide API for a node insertion to and removal from a backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey Shinkevich --- block.c | 49 + include/block/block.h | 3 +++ 2 files changed, 52 insertions(+) diff --git

[PATCH v12 00/14] Apply COR-filter to the block-stream permanently

2020-10-22 Thread Andrey Shinkevich via
The node insert/remove functions were added at the block generic layer. COR-filter options structure was added to the QAPI. The test case #310 was added to check the 'bottom' node limit for COR. The 'supported_read_flags' member was added to the BDS structure (with the flags check at the block

[PATCH v12 09/14] block: modify the comment for BDRV_REQ_PREFETCH flag

2020-10-22 Thread Andrey Shinkevich via
Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- include/block/block.h | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git

[PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver

2020-10-22 Thread Andrey Shinkevich via
We are going to use the COR-filter for a block-stream job. To limit COR operations by the base node in the backing chain during stream job, pass the bottom node name, that is the first non-filter overlay of the base, to the copy-on-read driver as the base node itself may change due to possible

[PATCH v12 13/14] stream: skip filters when writing backing file name to QCOW2 header

2020-10-22 Thread Andrey Shinkevich via
Avoid writing a filter JSON file name and a filter format name to QCOW2 image when the backing file is changed after the block stream job. A user is still able to assign the 'backing-file' parameter for a block-stream job keeping in mind the possible issue mentioned above. If the user does not

[PATCH v12 03/14] copy-on-read: add filter drop function

2020-10-22 Thread Andrey Shinkevich via
Provide API for the COR-filter removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. This function may be considered as an intermediate solution before we are able to use bdrv_remove_node(). It will be possible once the QEMU permission update

[PATCH v12 04/14] qapi: add filter-node-name to block-stream

2020-10-22 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4 +++-

[PATCH 0/2] Increase amount of data for monitor to read

2020-11-06 Thread Andrey Shinkevich via
The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A little parser is introduced to throttle JSON commands read from the buffer so that QMP requests do not overwhelm the

[PATCH 1/2] iotests: add another bash sleep command to 247

2020-11-06 Thread Andrey Shinkevich via
This patch paves the way for the one that follows. The following patch makes the QMP monitor to read up to 4K from stdin at once. That results in running the bash 'sleep' command before the _qemu_proc_exec() starts in subshell. Another 'sleep' command with an unobtrusive 'query-status' plays as a

[PATCH 2/2] monitor: increase amount of data for monitor to read

2020-11-06 Thread Andrey Shinkevich via
QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse. This patch increases the amount of read data up to 4096 bytes that fits the buffer size on the channel level.