[PATCH v13 10/10] block: apply COR-filter to block-stream jobs

2020-12-02 Thread Andrey Shinkevich via
jobs are about. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 97 ++ tests/qemu-iotests/030 | 51 +++- tests/qemu-iotests/030.out

[PATCH v13 03/10] copy-on-read: add filter drop function

2020-12-02 Thread Andrey Shinkevich via
as an intermediate solution before the QEMU permission update system has overhauled. Then we are able to implement the API function bdrv_remove_node() on the block generic layer. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 56

[PATCH v13 01/10] copy-on-read: support preadv/pwritev_part functions

2020-12-02 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions

[PATCH v13 02/10] block: add API function to insert a node

2020-12-02 Thread Andrey Shinkevich via
Provide API for insertion a node to backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block.c | 25 + include/block/block.h | 2 ++ 2 files changed, 27 insertions(+) diff --git

[PATCH v13 06/10] iotests: add #310 to test bottom node in COR driver

2020-12-02 Thread Andrey Shinkevich via
The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- tests/qemu-iotests/310 | 114

[PATCH v13 09/10] stream: skip filters when writing backing file name to QCOW2 header

2020-12-02 Thread Andrey Shinkevich via
keeping in mind the possible issue mentioned above. If the user does not specify the 'backing-file' parameter, QEMU will assign it automatically. Signed-off-by: Andrey Shinkevich --- block/stream.c | 21 +++-- blockdev.c | 8 +--- 2 files changed, 20 insertions(+), 9

[PATCH v13 07/10] block: include supported_read_flags into BDS structure

2020-12-02 Thread Andrey Shinkevich via
Add the new member supported_read_flags to the BlockDriverState structure. It will control the flags set for copy-on-read operations. Make the block generic layer evaluate supported read flags before they go to a block driver. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey

[PATCH v13 00/10] Apply COR-filter to the block-stream permanently

2020-12-02 Thread Andrey Shinkevich via
. 08: Three patches with fixes were merged into one. 09: The search for format_name of backing file was added. 10: The flag BLK_PERM_GRAPH_MOD was removed. Andrey Shinkevich (10): copy-on-read: support preadv/pwritev_part functions block: add API function to insert a node copy-on-rea

[PATCH v13 05/10] qapi: create BlockdevOptionsCor structure for COR driver

2020-12-02 Thread Andrey Shinkevich via
-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 57 ++-- qapi/block-core.json | 21 ++- 2 files changed, 75 insertions(+), 3 deletions(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c

[PATCH v13 08/10] copy-on-read: skip non-guest reads if no copy needed

2020-12-02 Thread Andrey Shinkevich via
-filter. block: Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 14 ++ include/block/block.h | 8 +--- 2 files changed, 15

[PATCH v13 04/10] qapi: add filter-node-name to block-stream

2020-12-02 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4

Re: [PATCH v12 14/14] block: apply COR-filter to block-stream jobs

2020-12-02 Thread Andrey Shinkevich
On 27.10.2020 21:24, Andrey Shinkevich wrote: On 27.10.2020 20:57, Vladimir Sementsov-Ogievskiy wrote: 27.10.2020 20:48, Andrey Shinkevich wrote: On 27.10.2020 19:13, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: This patch completes the series

[PATCH v3 3/5] monitor: let QMP monitor track JSON message content

2020-11-27 Thread Andrey Shinkevich via
a command into the queue. To keep that concept functional, the monitor should track the end of a single QMP command. It allows the dispatcher handling the command and send a response to client in time. Signed-off-by: Andrey Shinkevich --- include/qapi/qmp/json-parser.h | 5 +++-- monitor/qmp.c

[PATCH v3 5/5] monitor: increase amount of data for monitor to read

2020-11-27 Thread Andrey Shinkevich via
. Suggested-by: Denis V. Lunev Signed-off-by: Andrey Shinkevich --- monitor/monitor.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/monitor/monitor.c b/monitor/monitor.c index 84222cd..43d2d3b 100644 --- a/monitor/monitor.c +++ b/monitor/monitor.c @@ -566,7 +566,7 @@ int

[PATCH v3 2/5] monitor: drain requests queue with 'channel closed' event

2020-11-27 Thread Andrey Shinkevich via
When CHR_EVENT_CLOSED comes, the QMP requests queue may still contain unprocessed commands. It can happen with QMP capability OOB enabled. Let the dispatcher complete handling requests rest in the monitor queue. Signed-off-by: Andrey Shinkevich --- monitor/qmp.c | 46

Re: [PATCH v2 2/2] monitor: increase amount of data for monitor to read

2020-11-27 Thread Andrey Shinkevich
On 24.11.2020 14:03, Vladimir Sementsov-Ogievskiy wrote: 23.11.2020 18:44, Andrey Shinkevich wrote: QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse

[PATCH v3 0/5] Increase amount of data for monitor to read

2020-11-27 Thread Andrey Shinkevich via
xisting JSON parser were used to track the end of a QMP command. 04: The amount of read input data increases only. Andrey Shinkevich (4): monitor: change function obsolete name in comments monitor: drain requests queue with 'channel closed' event monitor: let QMP monitor track JSON message conte

[PATCH v3 4/5] iotests: 129 don't check backup "busy"

2020-11-27 Thread Andrey Shinkevich via
From: Vladimir Sementsov-Ogievskiy Busy is racy, job has it's "pause-points" when it's not busy. Drop this check. Signed-off-by: Vladimir Sementsov-Ogievskiy Reviewed-by: Max Reitz --- tests/qemu-iotests/129 | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/qemu-iotests/129

[PATCH v3 1/5] monitor: change function obsolete name in comments

2020-11-27 Thread Andrey Shinkevich via
The function name monitor_qmp_bh_dispatcher() has been changed to monitor_qmp_dispatcher_co() since the commit 9ce44e2c. Let's amend the comments. Signed-off-by: Andrey Shinkevich --- monitor/qmp.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/monitor/qmp.c b/monitor

Re: [PATCH v2 1/2] iotests: add another bash sleep command to 247

2020-11-24 Thread Andrey Shinkevich
On 23.11.2020 18:44, Andrey Shinkevich wrote: This patch paves the way for the one that follows. The following patch makes the QMP monitor to read up to 4K from stdin at once. That results in running the bash 'sleep' command before the _qemu_proc_exec() starts in subshell. Another 'sleep

Re: [PATCH v2 1/2] iotests: add another bash sleep command to 247

2020-11-24 Thread Andrey Shinkevich
On 24.11.2020 13:04, Vladimir Sementsov-Ogievskiy wrote: 23.11.2020 18:44, Andrey Shinkevich wrote: This patch paves the way for the one that follows. The following patch makes the QMP monitor to read up to 4K from stdin at once. That results in running the bash 'sleep' command before

Re: [PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich
On 23.11.2020 18:44, Andrey Shinkevich wrote: The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from

Re: [PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich
On 23.11.2020 18:44, Andrey Shinkevich wrote: The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution for the issue with QMP monitor buffered input. A JSON little parser is introduced to separate QMP commands read from

[PATCH v2 1/2] iotests: add another bash sleep command to 247

2020-11-23 Thread Andrey Shinkevich via
as a workaround. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/247 | 2 ++ tests/qemu-iotests/247.out | 1 + 2 files changed, 3 insertions(+) diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247 index 87e37b3..7d316ec 100755 --- a/tests/qemu-iotests/247 +++ b/tests/qemu-iotests

[PATCH v2 2/2] monitor: increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
. A JSON little parser is introduced to throttle QMP commands read from the buffer so that incoming requests do not overflow the monitor input queue. Suggested-by: Denis V. Lunev Signed-off-by: Andrey Shinkevich --- chardev/char-fd.c | 35 +-- chardev/char

[PATCH v2 0/2] Increase amount of data for monitor to read

2020-11-23 Thread Andrey Shinkevich via
synchronization was added to protect the input queue from overflow. Andrey Shinkevich (2): iotests: add another bash sleep command to 247 monitor: increase amount of data for monitor to read chardev/char-fd.c | 35 +-- chardev/char-soc

Re: [PATCH 2/2] monitor: increase amount of data for monitor to read

2020-11-16 Thread Andrey Shinkevich
On 09.11.2020 12:55, Vladimir Sementsov-Ogievskiy wrote: 06.11.2020 15:42, Andrey Shinkevich wrote: QMP and HMP monitors read one byte at a time from the socket or stdin, which is very inefficient. With 100+ VMs on the host, this results in multiple extra system calls and CPU overuse

Re: [PATCH 0/2] Increase amount of data for monitor to read

2020-11-09 Thread Andrey Shinkevich
On 09.11.2020 13:04, Vladimir Sementsov-Ogievskiy wrote: 09.11.2020 11:50, Vladimir Sementsov-Ogievskiy wrote: 06.11.2020 15:42, Andrey Shinkevich wrote: The subject was discussed here: https://lists.gnu.org/archive/html/qemu-devel/2017-05/msg00206.html This series is a solution

Re: [PATCH 0/2] Increase amount of data for monitor to read

2020-11-06 Thread Andrey Shinkevich
Please exclude this address when reply: jc...@redhat.com Andrey

[PATCH 2/2] monitor: increase amount of data for monitor to read

2020-11-06 Thread Andrey Shinkevich via
. Suggested-by: Denis V. Lunev Signed-off-by: Andrey Shinkevich --- chardev/char-fd.c | 64 +- chardev/char-socket.c | 54 +++--- chardev/char.c | 40 + include/chardev

[PATCH 1/2] iotests: add another bash sleep command to 247

2020-11-06 Thread Andrey Shinkevich via
as a workaround. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/247 | 2 ++ tests/qemu-iotests/247.out | 1 + 2 files changed, 3 insertions(+) diff --git a/tests/qemu-iotests/247 b/tests/qemu-iotests/247 index 87e37b3..7d316ec 100755 --- a/tests/qemu-iotests/247 +++ b/tests/qemu-iotests

[PATCH 0/2] Increase amount of data for monitor to read

2020-11-06 Thread Andrey Shinkevich via
#129 passed, the following patch should be applied first: '[PATCH v3 01/25] iotests: 129 don't check backup "busy"'. Andrey Shinkevich (2): iotests: add another bash sleep command to 247 monitor: increase amount of data for monitor to read chardev/char-fd.

Re: [PATCH v12 14/14] block: apply COR-filter to block-stream jobs

2020-10-27 Thread Andrey Shinkevich
On 27.10.2020 20:57, Vladimir Sementsov-Ogievskiy wrote: 27.10.2020 20:48, Andrey Shinkevich wrote: On 27.10.2020 19:13, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: This patch completes the series with the COR-filter insertion for block-stream operations

Re: [PATCH v12 14/14] block: apply COR-filter to block-stream jobs

2020-10-27 Thread Andrey Shinkevich
On 27.10.2020 19:13, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during

Re: [PATCH v12 13/14] stream: skip filters when writing backing file name to QCOW2 header

2020-10-27 Thread Andrey Shinkevich
On 27.10.2020 19:21, Vladimir Sementsov-Ogievskiy wrote: 27.10.2020 19:01, Andrey Shinkevich wrote: On 27.10.2020 18:09, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: Avoid writing a filter JSON file name and a filter format name to QCOW2 image when

Re: [PATCH v12 13/14] stream: skip filters when writing backing file name to QCOW2 header

2020-10-27 Thread Andrey Shinkevich
On 27.10.2020 18:09, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: Avoid writing a filter JSON file name and a filter format name to QCOW2 image when the backing file is changed after the block stream job. A user is still able to assign the 'backing-file

Re: [PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver

2020-10-23 Thread Andrey Shinkevich
On 23.10.2020 17:45, Vladimir Sementsov-Ogievskiy wrote: 22.10.2020 21:13, Andrey Shinkevich wrote: We are going to use the COR-filter for a block-stream job. To limit COR operations by the base node in the backing chain during stream job, pass the bottom node name, that is the first non

[PATCH v12 12/14] copy-on-read: skip non-guest reads if no copy needed

2020-10-22 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, skip idling read/write operations in COR-driver. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 12

[PATCH v12 06/14] copy-on-read: pass bottom node name to COR driver

2020-10-22 Thread Andrey Shinkevich via
concurrent jobs. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 16 1 file changed, 16 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 618c4c4..3d8e4db 100644

[PATCH v12 09/14] block: modify the comment for BDRV_REQ_PREFETCH flag

2020-10-22 Thread Andrey Shinkevich via
Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- include/block/block.h | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/include/block

[PATCH v12 13/14] stream: skip filters when writing backing file name to QCOW2 header

2020-10-22 Thread Andrey Shinkevich via
not specify the 'backing-file' parameter, QEMU will assign it automatically. Signed-off-by: Andrey Shinkevich --- block/stream.c | 15 +-- blockdev.c | 9 ++--- 2 files changed, 15 insertions(+), 9 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..1ba74ab

[PATCH v12 00/14] Apply COR-filter to the block-stream permanently

2020-10-22 Thread Andrey Shinkevich via
yer. 11: COR flag was added. 12: The condition was fixed. 13: The 'backing-file' parameter returned. No deprecation. 14: The COR-filter 'add' function replaced with the 'insert node' generic function. Fixes based on the review of the v11. Andrey Shinkevich (14): copy-on-read: supp

[PATCH v12 11/14] copy-on-read: add support for read flags to COR-filter

2020-10-22 Thread Andrey Shinkevich via
Add the BDRV_REQ_COPY_ON_READ and BDRV_REQ_PREFETCH flags to the supported_read_flags of the COR-filter. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 8178a91..a2b180a 100644

[PATCH v12 14/14] block: apply COR-filter to block-stream jobs

2020-10-22 Thread Andrey Shinkevich via
. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 98 ++ tests/qemu-iotests/030 | 51 +++- tests/qemu-iotests/030.out | 4 +- tests/qemu

[PATCH v12 01/14] copy-on-read: support preadv/pwritev_part functions

2020-10-22 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions

[PATCH v12 05/14] qapi: create BlockdevOptionsCor structure for COR driver

2020-10-22 Thread Andrey Shinkevich via
Create the BlockdevOptionsCor structure for COR driver specific options splitting it off form the BlockdevOptionsGenericFormat. The only option 'bottom' node in the structure denotes an image file that limits the COR operations in the backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey

[PATCH v12 10/14] block: include supported_read_flags into BDS structure

2020-10-22 Thread Andrey Shinkevich via
Add the new member supported_read_flags to the BlockDriverState structure. It will control the flags set for copy-on-read operations. Make the block generic layer evaluate supported read flags before they go to a block driver. Suggested-by: Vladimir Sementsov-Ogievskiy Signed-off-by: Andrey

[PATCH v12 02/14] block: add insert/remove node functions

2020-10-22 Thread Andrey Shinkevich via
Provide API for a node insertion to and removal from a backing chain. Suggested-by: Max Reitz Signed-off-by: Andrey Shinkevich --- block.c | 49 + include/block/block.h | 3 +++ 2 files changed, 52 insertions(+) diff --git

[PATCH v12 08/14] iotests: add #310 to test bottom node in COR driver

2020-10-22 Thread Andrey Shinkevich via
The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich --- tests/qemu-iotests/310 | 109 + tests/qemu-iotests/310.out | 15

[PATCH v12 07/14] copy-on-read: limit COR operations to bottom node

2020-10-22 Thread Andrey Shinkevich via
. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 42 -- 1 file changed, 40 insertions(+), 2 deletions(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index 3d8e4db..8178a91 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c

[PATCH v12 03/14] copy-on-read: add filter drop function

2020-10-22 Thread Andrey Shinkevich via
system has overhauled. To insert the filter, the block generic layer function bdrv_insert_node() can be used. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 56 block/copy-on-read.h | 32 ++ 2 files

[PATCH v12 04/14] qapi: add filter-node-name to block-stream

2020-10-22 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4

Re: [PATCH v11 09/13] copy-on-read: skip non-guest reads if no copy needed

2020-10-22 Thread Andrey Shinkevich
On 21.10.2020 23:43, Andrey Shinkevich wrote: On 14.10.2020 18:22, Vladimir Sementsov-Ogievskiy wrote: 14.10.2020 15:51, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can

Re: [PATCH v11 09/13] copy-on-read: skip non-guest reads if no copy needed

2020-10-21 Thread Andrey Shinkevich
On 14.10.2020 18:22, Vladimir Sementsov-Ogievskiy wrote: 14.10.2020 15:51, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms

Re: [PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-20 Thread Andrey Shinkevich
On 16.10.2020 18:45, Vladimir Sementsov-Ogievskiy wrote: 15.10.2020 20:16, Andrey Shinkevich wrote: On 14.10.2020 19:24, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: [...] ---   block/stream.c | 93 +-   tests/qemu

Re: [PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-16 Thread Andrey Shinkevich
On 15.10.2020 20:16, Andrey Shinkevich wrote: On 14.10.2020 19:24, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: [...] ---   block/stream.c | 93 +-   tests/qemu-iotests/030 | 51 +++--   tests

Re: [PATCH v11 05/13] copy-on-read: limit COR operations to base in COR driver

2020-10-15 Thread Andrey Shinkevich
On 15.10.2020 18:56, Max Reitz wrote: On 14.10.20 20:57, Andrey Shinkevich wrote: On 14.10.2020 15:01, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Limit COR operations by the base node in the backing chain when the overlay base node name is given. It will be useful

Re: [PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-15 Thread Andrey Shinkevich
On 14.10.2020 19:24, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: [...] --- block/stream.c | 93 +- tests/qemu-iotests/030 | 51 +++-- tests/qemu-iotests/030.out | 4 +- tests/qemu-iotests

Re: [PATCH v11 11/13] stream: mark backing-file argument as deprecated

2020-10-15 Thread Andrey Shinkevich
On 14.10.2020 18:43, Vladimir Sementsov-Ogievskiy wrote: 14.10.2020 18:03, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Whereas the block-stream job starts using a backing file name of the base node overlay after the block-stream job completes, mark the QMP 'backing-file

Re: [PATCH v11 09/13] copy-on-read: skip non-guest reads if no copy needed

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 15:51, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream

Re: [PATCH v11 06/13] block: modify the comment for BDRV_REQ_PREFETCH flag

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 15:22, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- include/block/block.h | 7

Re: [PATCH v11 05/13] copy-on-read: limit COR operations to base in COR driver

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 15:01, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Limit COR operations by the base node in the backing chain when the overlay base node name is given. It will be useful for a block stream job when the COR-filter is applied. The overlay base node is passed

Re: [PATCH v11 05/13] copy-on-read: limit COR operations to base in COR driver

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 14:59, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Limit COR operations by the base node in the backing chain when the overlay base node name is given. It will be useful for a block stream job when the COR-filter is applied. The overlay base node is passed

Re: [PATCH v11 04/13] copy-on-read: pass overlay base node name to COR driver

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 14:09, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: We are going to use the COR-filter for a block-stream job. To limit COR operations by the base node in the backing chain during stream job, pass the name of overlay base node to the copy-on-read driver as base

Re: [PATCH v11 02/13] copy-on-read: add filter append/drop functions

2020-10-14 Thread Andrey Shinkevich
On 14.10.2020 13:44, Max Reitz wrote: On 12.10.20 19:43, Andrey Shinkevich wrote: Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir

[PATCH v11 13/13] block: apply COR-filter to block-stream jobs

2020-10-12 Thread Andrey Shinkevich via
. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 93 +- tests/qemu-iotests/030 | 51 +++-- tests/qemu-iotests/030.out | 4 +- tests/qemu

[PATCH v11 12/13] stream: remove unused backing-file name parameter

2020-10-12 Thread Andrey Shinkevich via
The 'backing-file' argument is not used by the block-stream job. It designates a backing file name to set in QCOW2 image header after the block-stream job finished. A backing file name of the node above base is used instead. Signed-off-by: Andrey Shinkevich --- block/stream.c| 6

[PATCH v11 11/13] stream: mark backing-file argument as deprecated

2020-10-12 Thread Andrey Shinkevich via
Whereas the block-stream job starts using a backing file name of the base node overlay after the block-stream job completes, mark the QMP 'backing-file' argument as deprecated. Signed-off-by: Andrey Shinkevich --- docs/system/deprecated.rst | 6 ++ 1 file changed, 6 insertions(+) diff

[PATCH v11 08/13] copy-on-read: add support for BDRV_REQ_PREFETCH to COR-filter

2020-10-12 Thread Andrey Shinkevich via
Add support for the BDRV_REQ_PREFETCH flag to the supported_write_flags of the COR-filter. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index dfbd6ad..b136895 100644 --- a/block/copy

[PATCH v11 09/13] copy-on-read: skip non-guest reads if no copy needed

2020-10-12 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 13

[PATCH v11 10/13] stream: skip filters when writing backing file name to QCOW2 header

2020-10-12 Thread Andrey Shinkevich via
Avoid writing a filter JSON-name to QCOW2 image when the backing file is changed after the block stream job. Signed-off-by: Andrey Shinkevich --- block/stream.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..51462bd

[PATCH v11 07/13] block: include supported_read_flags into BDS structure

2020-10-12 Thread Andrey Shinkevich via
Add the new member supported_read_flags to BlockDriverState structure. It will control the BDRV_REQ_PREFETCH flag set for copy-on-read operations. Signed-off-by: Andrey Shinkevich --- include/block/block_int.h | 4 1 file changed, 4 insertions(+) diff --git a/include/block/block_int.h b

[PATCH v11 06/13] block: modify the comment for BDRV_REQ_PREFETCH flag

2020-10-12 Thread Andrey Shinkevich via
Modify the comment for the flag BDRV_REQ_PREFETCH as we are going to use it alone and pass it to the COR-filter driver for further processing. Signed-off-by: Andrey Shinkevich --- include/block/block.h | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/include/block

[PATCH v11 05/13] copy-on-read: limit COR operations to base in COR driver

2020-10-12 Thread Andrey Shinkevich via
. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 39 +-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index c578b1b..dfbd6ad 100644 --- a/block/copy-on-read.c +++ b/block/copy-on-read.c

[PATCH v11 02/13] copy-on-read: add filter append/drop functions

2020-10-12 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 88

[PATCH v11 04/13] copy-on-read: pass overlay base node name to COR driver

2020-10-12 Thread Andrey Shinkevich via
will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 14 ++ 1 file changed, 14 insertions(+) diff --git a/block/copy-on-read.c b/block/copy-on-read.c index bcccf0f..c578b1b 100644 --- a/block/copy-on-read.c +++ b/block/copy

[PATCH v11 03/13] qapi: add filter-node-name to block-stream

2020-10-12 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4

[PATCH v11 01/13] copy-on-read: Support preadv/pwritev_part functions

2020-10-12 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions

[PATCH v11 00/13] Apply COR-filter to the block-stream permanently

2020-10-12 Thread Andrey Shinkevich via
the stream_populate(); The 'implicit' initialization moved back to COR-filter driver. Base node overlay is used instead of base. The v8 Message-Id: <1601383109-110988-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (13): copy-on-read: Support preadv/pwrite

Re: [PATCH v10 6/9] copy-on-read: skip non-guest reads if no copy needed

2020-10-09 Thread Andrey Shinkevich
On 07.10.2020 22:28, Vladimir Sementsov-Ogievskiy wrote: 07.10.2020 22:01, Andrey Shinkevich wrote: On 07.10.2020 13:06, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded

Re: [PATCH v10 8/9] block: remove unused backing-file name parameter

2020-10-07 Thread Andrey Shinkevich
On 07.10.2020 13:21, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: The block stream QMP parameter backing-file is in use no more. It designates a backing file name to set in QCOW2 image header after the block stream job finished. The base file name is used

Re: [PATCH v10 6/9] copy-on-read: skip non-guest reads if no copy needed

2020-10-07 Thread Andrey Shinkevich
On 07.10.2020 13:06, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made

Re: [PATCH v10 9/9] block: apply COR-filter to block-stream jobs

2020-10-07 Thread Andrey Shinkevich
On 07.10.2020 20:27, Andrey Shinkevich wrote: On 29.09.2020 15:38, Andrey Shinkevich wrote: This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block

Re: [PATCH v10 9/9] block: apply COR-filter to block-stream jobs

2020-10-07 Thread Andrey Shinkevich
On 29.09.2020 15:38, Andrey Shinkevich wrote: This patch completes the series with the COR-filter insertion for block-stream operations. Adding the filter makes it possible for copied regions to be discarded in backing files during the block-stream job, what will reduce the disk overuse

Re: [PATCH v10 5/9] copy-on-read: limit guest COR activity to base in COR driver

2020-10-05 Thread Andrey Shinkevich
On 05.10.2020 17:58, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: Limit the guest's COR operations by the base node in the backing chain when the base node name is given. It will be useful for a block stream job when the COR-filter is applied. Signed-off

Re: [PATCH v10 5/9] copy-on-read: limit guest COR activity to base in COR driver

2020-10-05 Thread Andrey Shinkevich
On 05.10.2020 17:58, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: Limit the guest's COR operations by the base node in the backing chain when the base node name is given. It will be useful for a block stream job when the COR-filter is applied. Signed-off

Re: [PATCH v10 2/9] copy-on-read: add filter append/drop functions

2020-10-05 Thread Andrey Shinkevich
On 05.10.2020 16:34, Vladimir Sementsov-Ogievskiy wrote: 29.09.2020 15:38, Andrey Shinkevich wrote: Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich

[PATCH v10 7/9] stream: skip filters when writing backing file name to QCOW2 header

2020-09-29 Thread Andrey Shinkevich via
Avoid writing a filter JSON-name to QCOW2 image when the backing file is changed after the block stream job. Signed-off-by: Andrey Shinkevich --- block/stream.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/block/stream.c b/block/stream.c index e0540ee..b0719e9

[PATCH v10 9/9] block: apply COR-filter to block-stream jobs

2020-09-29 Thread Andrey Shinkevich via
. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 93 ++ tests/qemu-iotests/030 | 51 +++-- tests/qemu-iotests/030.out | 4 +- tests/qemu

[PATCH v10 6/9] copy-on-read: skip non-guest reads if no copy needed

2020-09-29 Thread Andrey Shinkevich via
If the flag BDRV_REQ_PREFETCH was set, pass it further to the COR-driver to skip unneeded reading. It can be taken into account for the COR-algorithms optimization. That check is being made during the block stream job by the moment. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 14

[PATCH v10 4/9] copy-on-read: pass base node name to COR driver

2020-09-29 Thread Andrey Shinkevich via
To limit the guest's COR operations by the base node in the backing chain during stream job, pass the base node name to the copy-on-read driver. The rest of the functionality will be implemented in the patch that follows. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 13

[PATCH v10 8/9] block: remove unused backing-file name parameter

2020-09-29 Thread Andrey Shinkevich via
The block stream QMP parameter backing-file is in use no more. It designates a backing file name to set in QCOW2 image header after the block stream job finished. The base file name is used instead. Signed-off-by: Andrey Shinkevich --- block/monitor/block-hmp-cmds.c | 2 +- block/stream.c

[PATCH v10 3/9] qapi: add filter-node-name to block-stream

2020-09-29 Thread Andrey Shinkevich via
Provide the possibility to pass the 'filter-node-name' parameter to the block-stream job as it is done for the commit block job. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/monitor/block-hmp-cmds.c | 4 ++-- block/stream.c | 4

[PATCH v10 2/9] copy-on-read: add filter append/drop functions

2020-09-29 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 84 block/copy-on-read.h | 35

[PATCH v10 1/9] copy-on-read: Support preadv/pwritev_part functions

2020-09-29 Thread Andrey Shinkevich via
Add support for the recently introduced functions bdrv_co_preadv_part() and bdrv_co_pwritev_part() to the COR-filter driver. Signed-off-by: Andrey Shinkevich Reviewed-by: Vladimir Sementsov-Ogievskiy --- block/copy-on-read.c | 28 1 file changed, 16 insertions

[PATCH v10 0/9] Apply COR-filter to the block-stream permanently

2020-09-29 Thread Andrey Shinkevich via
221780-1-git-send-email-andrey.shinkev...@virtuozzo.com> Andrey Shinkevich (9): copy-on-read: Support preadv/pwritev_part functions copy-on-read: add filter append/drop functions qapi: add filter-node-name to block-stream copy-on-read: pass base node name to COR driver copy-on-read: limit

[PATCH v10 5/9] copy-on-read: limit guest COR activity to base in COR driver

2020-09-29 Thread Andrey Shinkevich via
Limit the guest's COR operations by the base node in the backing chain when the base node name is given. It will be useful for a block stream job when the COR-filter is applied. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 38 -- 1 file changed

[PATCH v9 9/9] block: apply COR-filter to block-stream jobs

2020-09-28 Thread Andrey Shinkevich via
. The concept of the parallel jobs with common nodes is considered vital no more. Signed-off-by: Andrey Shinkevich --- block/stream.c | 93 ++ tests/qemu-iotests/030 | 51 +++-- tests/qemu-iotests/030.out | 4 +- tests/qemu

[PATCH v9 8/9] block: remove unused backing-file name parameter

2020-09-28 Thread Andrey Shinkevich via
The block stream QMP parameter backing-file is in use no more. It designates a backing file name to set in QCOW2 image header after the block stream job finished. The base file name is used instead. Signed-off-by: Andrey Shinkevich --- block/monitor/block-hmp-cmds.c | 2 +- block/stream.c

[PATCH v9 2/9] copy-on-read: add filter append/drop functions

2020-09-28 Thread Andrey Shinkevich via
Provide API for the COR-filter insertion/removal. Also, drop the filter child permissions for an inactive state when the filter node is being removed. Signed-off-by: Andrey Shinkevich --- block/copy-on-read.c | 84 1 file changed, 84

  1   2   3   4   5   6   7   8   9   >