Re: [Qemu-block] [Qemu-devel] [PATCH v2 00/20] dataplane: remove RFifoLock

On Wed, 10/19 08:55, Fam Zheng wrote: > Reviewed-by: Fam Zheng FWIW in merging it: Acked-by: Fam Zheng

Re: [Qemu-block] [PATCH v12 10/19] block: Add QMP support for streaming to an intermediate layer

On Wed 26 Oct 2016 04:58:00 PM CEST, Kevin Wolf wrote: > Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: >> This patch makes the 'device' parameter of the 'block-stream' command >> accept a node name that is not a root node. >> >> In addition to that, operation blockers

Re: [Qemu-block] [Qemu-devel] [PATCH 0/7] blockjobs: preliminary refactoring work, Pt 1

On 10/26/2016 12:52 AM, Jeff Cody wrote: On Fri, Oct 14, 2016 at 02:32:55PM -0400, John Snow wrote: On 10/13/2016 06:56 PM, John Snow wrote: This is a follow-up to patches 1-6 of: [PATCH v2 00/11] blockjobs: Fix transactional race condition That series started trying to refactor blockjobs

Re: [Qemu-block] [Qemu-devel] [PATCH v8 00/36] block: Image locking series

On 26.10.2016 17:33, Kevin Wolf wrote: > Am 26.10.2016 um 17:12 hat Max Reitz geschrieben: >> On 26.10.2016 13:01, Fam Zheng wrote: >>> On Tue, 10/25 16:57, Kevin Wolf wrote: Am 25.10.2016 um 15:30 hat Max Reitz geschrieben: > On 25.10.2016 10:24, Kevin Wolf wrote: >> Am 24.10.2016 um

Re: [Qemu-block] [Qemu-devel] [PATCH v8 00/36] block: Image locking series

Am 26.10.2016 um 17:12 hat Max Reitz geschrieben: > On 26.10.2016 13:01, Fam Zheng wrote: > > On Tue, 10/25 16:57, Kevin Wolf wrote: > >> Am 25.10.2016 um 15:30 hat Max Reitz geschrieben: > >>> On 25.10.2016 10:24, Kevin Wolf wrote: > Am 24.10.2016 um 20:03 hat Max Reitz geschrieben: > >

Re: [Qemu-block] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

On 26.10.2016 11:04, Vladimir Sementsov-Ogievskiy wrote: > 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote: >> 24.10.2016 20:18, Max Reitz wrote: >>> On 24.10.2016 19:08, Max Reitz wrote: On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote: > 24.10.2016 13:32, Vladimir

Re: [Qemu-block] [Qemu-devel] [PATCH v8 00/36] block: Image locking series

On 26.10.2016 13:01, Fam Zheng wrote: > On Tue, 10/25 16:57, Kevin Wolf wrote: >> Am 25.10.2016 um 15:30 hat Max Reitz geschrieben: >>> On 25.10.2016 10:24, Kevin Wolf wrote: Am 24.10.2016 um 20:03 hat Max Reitz geschrieben: > On 24.10.2016 12:11, Kevin Wolf wrote: > > [...] >

Re: [Qemu-block] [PATCH v8 00/36] block: Image locking series

On 25.10.2016 16:57, Kevin Wolf wrote: > Am 25.10.2016 um 15:30 hat Max Reitz geschrieben: >> On 25.10.2016 10:24, Kevin Wolf wrote: >>> Am 24.10.2016 um 20:03 hat Max Reitz geschrieben: On 24.10.2016 12:11, Kevin Wolf wrote: [...] > Now, the big question is how to

Re: [Qemu-block] [PATCH v12 11/19] docs: Document how to stream to an intermediate layer

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > Signed-off-by: Alberto Garcia Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v12 10/19] block: Add QMP support for streaming to an intermediate layer

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > This patch makes the 'device' parameter of the 'block-stream' command > accept a node name that is not a root node. > > In addition to that, operation blockers will be checked in all > intermediate nodes between the top and the base node. >

Re: [Qemu-block] [PATCH v8 05/36] raw-posix: Add image locking support

On 25.10.2016 15:43, Fam Zheng wrote: > On Tue, 10/25 15:28, Max Reitz wrote: >> On 25.10.2016 08:31, Fam Zheng wrote: >>> On Sat, 10/22 01:40, Max Reitz wrote: On 30.09.2016 14:09, Fam Zheng wrote: >> >> [...] >> > +static int > +raw_reopen_upgrade(BDRVReopenState *state, > +

Re: [Qemu-block] [PATCH v12 09/19] block: Support streaming to an intermediate layer

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > This makes sure that the image we are streaming into is open in > read-write mode during the operation. > > Operation blockers are also set in all intermediate nodes, since they > will be removed from the chain afterwards. > > Finally,

Re: [Qemu-block] [PATCH v12 07/19] block: Block all nodes involved in the block-commit operation

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > After a successful block-commit operation all nodes between top and > base are removed from the backing chain, and top's overlay needs to > be updated to point to base. Because of that we should prevent other > block jobs from messing with

Re: [Qemu-block] [Qemu-devel] [PATCH 2/4] block/curl: Fix return value from curl_read_cb

On 25.10.2016 20:37, Eric Blake wrote: > On 10/24/2016 09:54 PM, Max Reitz wrote: >> While commit 38bbc0a580f9f10570b1d1b5d3e92f0e6feb2970 is correct in that >> the callback is supposed to return the number of bytes handled; what it >> does not mention is that libcurl will throw an error if the

Re: [Qemu-block] [PATCH v12 04/19] block: Use block_job_add_bdrv() in mirror_start_job()

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > Use block_job_add_bdrv() instead of blocking all operations in > mirror_start_job() and unblocking them in mirror_exit(). > > Signed-off-by: Alberto Garcia Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v12 05/19] block: Use block_job_add_bdrv() in backup_start()

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > Use block_job_add_bdrv() instead of blocking all operations in > backup_start() and unblocking them in backup_run(). > > Signed-off-by: Alberto Garcia Reviewed-by: Kevin Wolf

Re: [Qemu-block] [Qemu-devel] [PATCH 1/4] block/curl: Use BDRV_SECTOR_SIZE

On 25.10.2016 20:31, Eric Blake wrote: > On 10/24/2016 09:54 PM, Max Reitz wrote: >> Currently, curl defines its own constant SECTOR_SIZE. There is no >> advantage over using the global BDRV_SECTOR_SIZE, so drop it. >> >> Cc: qemu-sta...@nongnu.org >> Signed-off-by: Max Reitz

Re: [Qemu-block] [PATCH v12 02/19] block: Pause all jobs during bdrv_reopen_multiple()

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > When a BlockDriverState is about to be reopened it can trigger certain > operations that need to write to disk. During this process a different > block job can be woken up. If that block job completes and also needs > to call bdrv_reopen()

Re: [Qemu-block] [PATCH v12 01/19] block: Add bdrv_drain_all_{begin, end}()

Am 26.10.2016 um 12:29 hat Alberto Garcia geschrieben: > bdrv_drain_all() doesn't allow the caller to do anything after all > pending requests have been completed but before block jobs are > resumed. > > This patch splits bdrv_drain_all() into _begin() and _end() for that > purpose. It also adds

Re: [Qemu-block] [PATCH v5 12/13] iotests: Add assert_json_filename_equal() method

On 26.10.2016 12:41, Kevin Wolf wrote: > Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: >> Since the order of keys in JSON filenames is not necessarily fixed, they >> should not be compared to fixed strings. This method takes a Python dict >> as a reference, parses a given JSON filename and

Re: [Qemu-block] [Qemu-devel] [PATCH v5 13/13] iotests: Add test for NBD's blockdev-add interface

On 25.10.2016 16:41, Markus Armbruster wrote: > Max Reitz writes: > >> Signed-off-by: Max Reitz >> --- >> tests/qemu-iotests/147 | 196 >> + >> tests/qemu-iotests/147.out | 5 ++ >>

Re: [Qemu-block] [PATCH v5 08/13] qapi: Allow blockdev-add for NBD

On 26.10.2016 01:57, Eric Blake wrote: > On 10/25/2016 08:11 AM, Max Reitz wrote: >> Signed-off-by: Max Reitz >> --- >> qapi/block-core.json | 25 ++--- >> 1 file changed, 22 insertions(+), 3 deletions(-) >> >> diff --git a/qapi/block-core.json

Re: [Qemu-block] [PATCH v4 5/5] qapi: allow blockdev-add for ssh

On 10/26/2016 03:31 AM, Kevin Wolf wrote: >>> +# @host_key_check #optional defines how and what to check the host >>> +# key against, defaults to "yes" >> >> I still have reservations about this parameter. > > It doesn't seem to be as easy as an enum. The real thing

Re: [Qemu-block] [PATCH v5 13/13] iotests: Add test for NBD's blockdev-add interface

Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz With the trailing newline fixed that Markus mentioned: Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

26.10.2016 15:13, Vladimir Sementsov-Ogievskiy wrote: 26.10.2016 12:21, Vladimir Sementsov-Ogievskiy wrote: 26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote: 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 20:18, Max Reitz wrote: On 24.10.2016 19:08, Max Reitz wrote: On

Re: [Qemu-block] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

26.10.2016 12:21, Vladimir Sementsov-Ogievskiy wrote: 26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote: 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 20:18, Max Reitz wrote: On 24.10.2016 19:08, Max Reitz wrote: On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote:

Re: [Qemu-block] [Qemu-devel] [PATCH v8 00/36] block: Image locking series

On Tue, 10/25 16:57, Kevin Wolf wrote: > Am 25.10.2016 um 15:30 hat Max Reitz geschrieben: > > On 25.10.2016 10:24, Kevin Wolf wrote: > > > Am 24.10.2016 um 20:03 hat Max Reitz geschrieben: > > >> On 24.10.2016 12:11, Kevin Wolf wrote: > > >> > > >> [...] > > >> > > >>> Now, the big question is

[Qemu-block] [PATCH v12 12/19] qemu-iotests: Test streaming to an intermediate layer

This adds test_stream_intermediate(), similar to test_stream() but streams to the intermediate image instead. It also removes the usage of blkdebug, which is unnecessary for this test. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/030 | 21 -

[Qemu-block] [PATCH v12 07/19] block: Block all nodes involved in the block-commit operation

After a successful block-commit operation all nodes between top and base are removed from the backing chain, and top's overlay needs to be updated to point to base. Because of that we should prevent other block jobs from messing with them. This patch blocks all operations in these nodes in

[Qemu-block] [PATCH v12 00/19] Support streaming to an intermediate layer

Hi all, here's version 12 of the series. The problem with version 11 and the bdrv_drain_all() split is that it won't work if new block jobs are created or new BDSs are added between the _begin() and _end() calls. We were discussing this on the mailing list and on IRC. Adding a global quiesce

[Qemu-block] [PATCH v12 15/19] qemu-iotests: Test block-stream and block-commit in parallel

As with test_stream_parallel(), we allow mixing block-stream and block-commit operations in the same backing chain as long as there's no overlap among the involved nodes. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/030 | 30 ++

[Qemu-block] [PATCH v12 03/19] block: Add block_job_add_bdrv()

When a block job is created on a certain BlockDriverState, operations are blocked there while the job exists. However, some block jobs may involve additional BDSs, which must be blocked separately when the job is created and unblocked manually afterwards. This patch adds block_job_add_bdrv(),

[Qemu-block] [PATCH v12 04/19] block: Use block_job_add_bdrv() in mirror_start_job()

Use block_job_add_bdrv() instead of blocking all operations in mirror_start_job() and unblocking them in mirror_exit(). Signed-off-by: Alberto Garcia --- block/mirror.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/block/mirror.c b/block/mirror.c index

[Qemu-block] [PATCH v12 18/19] block: Add 'base-node' parameter to the 'block-stream' command

The way to specify the node from which to copy data in the block-stream operation is by using the 'base' parameter. This parameter however takes a file name, not a node name. Since we want to be able to perform this operation using only node names, this patch adds a new 'base-node' parameter.

[Qemu-block] [PATCH v12 10/19] block: Add QMP support for streaming to an intermediate layer

This patch makes the 'device' parameter of the 'block-stream' command accept a node name that is not a root node. In addition to that, operation blockers will be checked in all intermediate nodes between the top and the base node. Signed-off-by: Alberto Garcia --- blockdev.c

[Qemu-block] [PATCH v12 11/19] docs: Document how to stream to an intermediate layer

Signed-off-by: Alberto Garcia --- docs/live-block-ops.txt | 36 +--- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/docs/live-block-ops.txt b/docs/live-block-ops.txt index a257087..2211d14 100644 --- a/docs/live-block-ops.txt +++

[Qemu-block] [PATCH v12 05/19] block: Use block_job_add_bdrv() in backup_start()

Use block_job_add_bdrv() instead of blocking all operations in backup_start() and unblocking them in backup_run(). Signed-off-by: Alberto Garcia --- block/backup.c | 5 + 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/block/backup.c b/block/backup.c index

[Qemu-block] [PATCH v12 16/19] qemu-iotests: Add iotests.supports_quorum()

There's many tests that need Quorum support in order to run. At the moment each test implements its own check to see if Quorum is enabled. This patch centralizes all those checks in a new function called iotests.supports_quorum(). Signed-off-by: Alberto Garcia ---

[Qemu-block] [PATCH v12 19/19] qemu-iotests: Test the 'base-node' parameter of 'block-stream'

The block-stream command has traditionally used the 'base' parameter to indicate the image to copy the data from. This test checks that the 'base-node' parameter can also be used for the same purpose. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/030 | 33

Re: [Qemu-block] [PATCH v5 12/13] iotests: Add assert_json_filename_equal() method

Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: > Since the order of keys in JSON filenames is not necessarily fixed, they > should not be compared to fixed strings. This method takes a Python dict > as a reference, parses a given JSON filename and compares both. > > Signed-off-by: Max Reitz

[Qemu-block] [PATCH v12 17/19] qemu-iotests: Test streaming to a Quorum child

Quorum children are special in the sense that they're not directly attached to a block backend but they're not used as backing images either. However the intermediate block streaming code supports streaming to them. This is a test case for that scenario. Signed-off-by: Alberto Garcia

[Qemu-block] [PATCH v12 06/19] block: Check blockers in all nodes involved in a block-commit job

qmp_block_commit() checks for op blockers in the active and destination (base) images. However all nodes between top_bs and base are also involved, and they are removed from the chain afterwards. In addition to that, if top_bs is not the active layer then top_bs's overlay also needs to be checked

[Qemu-block] [PATCH v12 09/19] block: Support streaming to an intermediate layer

This makes sure that the image we are streaming into is open in read-write mode during the operation. Operation blockers are also set in all intermediate nodes, since they will be removed from the chain afterwards. Finally, this also unblocks the stream operation in backing files.

[Qemu-block] [PATCH v12 08/19] block: Block all intermediate nodes in commit_active_start()

When block-commit is launched without the top parameter, it uses internally a mirror block job. In that case all intermediate nodes between the active and base nodes must be blocked as well. Signed-off-by: Alberto Garcia Reviewed-by: Kevin Wolf ---

[Qemu-block] [PATCH v12 14/19] qemu-iotests: Test overlapping stream and commit operations

These test cases check that it's not possible to perform two block-stream or block-commit operations if there are nodes involved in both. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/030 | 89 ++

[Qemu-block] [PATCH v12 01/19] block: Add bdrv_drain_all_{begin, end}()

bdrv_drain_all() doesn't allow the caller to do anything after all pending requests have been completed but before block jobs are resumed. This patch splits bdrv_drain_all() into _begin() and _end() for that purpose. It also adds aio_{disable,enable}_external() calls to disable external clients

Re: [Qemu-block] [PATCH v5 09/13] iotests.py: Add qemu_nbd function

Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v5 08/13] qapi: Allow blockdev-add for NBD

Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v5 07/13] block/nbd: Use SocketAddress options

Am 25.10.2016 um 15:11 hat Max Reitz geschrieben: > Drop the use of legacy options in favor of the SocketAddress > representation, even for internal use (i.e. for storing the result of > the filename parsing). > > Signed-off-by: Max Reitz Reviewed-by: Kevin Wolf

Re: [Qemu-block] [PATCH v2 2/2] qapi: allow blockdev-add for NFS

Am 25.10.2016 um 23:16 hat Eric Blake geschrieben: > On 10/24/2016 02:27 PM, Ashijeet Acharya wrote: > > Introduce new object 'BlockdevOptionsNFS' in qapi/block-core.json to > > support blockdev-add for NFS network protocol driver. Also make a new > > struct NFSServer to support tcp connection. >

Re: [Qemu-block] [Qemu-devel] [PATCH 0/4] block/curl: Fix FTP

As well as testing patch 2 on its own, I also tested all 4 patches together on top of current qemu from git. In summary, it seems to work fine and doesn't break http or https as far as I can tell. Rich. $ http_proxy= LIBGUESTFS_BACKEND=direct

Re: [Qemu-block] [Qemu-devel] [PATCH 2/4] block/curl: Fix return value from curl_read_cb

On Tue, Oct 25, 2016 at 04:54:29AM +0200, Max Reitz wrote: > While commit 38bbc0a580f9f10570b1d1b5d3e92f0e6feb2970 is correct in that > the callback is supposed to return the number of bytes handled; what it > does not mention is that libcurl will throw an error if the callback did > not "handle"

Re: [Qemu-block] [Qemu-devel] [PATCH 1/4] block/curl: Use BDRV_SECTOR_SIZE

On Tue, Oct 25, 2016 at 04:54:28AM +0200, Max Reitz wrote: > Currently, curl defines its own constant SECTOR_SIZE. There is no > advantage over using the global BDRV_SECTOR_SIZE, so drop it. > > Cc: qemu-sta...@nongnu.org > Signed-off-by: Max Reitz > --- > block/curl.c | 11

Re: [Qemu-block] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

26.10.2016 12:04, Vladimir Sementsov-Ogievskiy wrote: 25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 20:18, Max Reitz wrote: On 24.10.2016 19:08, Max Reitz wrote: On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет:

Re: [Qemu-block] [Qemu-devel] [PATCH 2/4] block/curl: Fix return value from curl_read_cb

Am 25.10.2016 um 20:37 hat Eric Blake geschrieben: > On 10/24/2016 09:54 PM, Max Reitz wrote: > > While commit 38bbc0a580f9f10570b1d1b5d3e92f0e6feb2970 is correct in that > > the callback is supposed to return the number of bytes handled; what it > > does not mention is that libcurl will throw an

Re: [Qemu-block] [PATCH 12/22] qcow2-bitmap: add IN_USE flag

25.10.2016 13:53, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 20:18, Max Reitz wrote: On 24.10.2016 19:08, Max Reitz wrote: On 24.10.2016 13:35, Vladimir Sementsov-Ogievskiy wrote: 24.10.2016 13:32, Vladimir Sementsov-Ogievskiy пишет: 21.10.2016 22:58, Max Reitz пишет: On 21.10.2016

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/2] qapi: allow blockdev-add for NFS

Am 26.10.2016 um 09:23 hat Markus Armbruster geschrieben: > A few drive-by comments... > > Eric Blake writes: > > > On 10/24/2016 02:27 PM, Ashijeet Acharya wrote: > >> Introduce new object 'BlockdevOptionsNFS' in qapi/block-core.json to > >> support blockdev-add for NFS

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/2] qapi: allow blockdev-add for NFS

Am 26.10.2016 um 10:40 hat Markus Armbruster geschrieben: > Kevin Wolf writes: > > > Am 26.10.2016 um 09:23 hat Markus Armbruster geschrieben: > >> A few drive-by comments... > >> > >> Eric Blake writes: > >> > >> > On 10/24/2016 02:27 PM, Ashijeet Acharya

Re: [Qemu-block] [Qemu-devel] [PATCH v2 2/2] qapi: allow blockdev-add for NFS

Kevin Wolf writes: > Am 26.10.2016 um 09:23 hat Markus Armbruster geschrieben: >> A few drive-by comments... >> >> Eric Blake writes: >> >> > On 10/24/2016 02:27 PM, Ashijeet Acharya wrote: >> >> Introduce new object 'BlockdevOptionsNFS' in

Re: [Qemu-block] [PATCH v4 5/5] qapi: allow blockdev-add for ssh

Am 25.10.2016 um 21:58 hat Eric Blake geschrieben: > On 10/25/2016 08:04 AM, Ashijeet Acharya wrote: > > Introduce new object 'BlockdevOptionsSsh' in qapi/block-core.json to > > support blockdev-add for SSH network protocol driver. Use only 'struct > > InetSocketAddress' since SSH only supports