[Qemu-block] [PATCH 7/8] qemu-iotests: fix test_stream_partial()

This test is streaming to the top layer using the intermediate image as the base. This is a mistake since block-stream never copies data from the base image and its backing chain, so this is effectively a no-op. In addition to fixing the base parameter, this patch also writes some data to the inte

[Qemu-block] [PATCH 8/8] qemu-iotests: test streaming to an intermediate layer

This adds test_stream_intermediate(), similar to test_stream() but streams to the intermediate image instead. Signed-off-by: Alberto Garcia --- tests/qemu-iotests/030 | 18 +- tests/qemu-iotests/030.out | 4 ++-- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git

[Qemu-block] [PATCH 6/8] docs: Document how to stream to an intermediate layer

Signed-off-by: Alberto Garcia --- docs/live-block-ops.txt | 30 +++--- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/docs/live-block-ops.txt b/docs/live-block-ops.txt index a257087..3bf86be 100644 --- a/docs/live-block-ops.txt +++ b/docs/live-block-ops.tx

[Qemu-block] [PATCH 2/8] block: allow block jobs in any arbitrary node

Currently, block jobs can only be owned by root nodes. This patch allows block jobs to be in any arbitrary node, by making the following changes: - Block jobs can now be identified by the node name of their BlockDriverState in addition to the device name. Since both device and node names live

[Qemu-block] [PATCH 4/8] block: Support streaming to an intermediate layer

This makes sure that the image we are steaming into is open in read-write mode during the operation. Operation blockers are also set in all intermediate nodes, since they will be removed from the chain afterwards. Finally, this also unblocks the stream operation in backing files. Signed-off-by:

[Qemu-block] [PATCH 5/8] block: Add QMP support for streaming to an intermediate layer

This patch makes the 'device' paramater of the 'block-stream' command accept a node name as well as a device name. In addition to that, operation blockers will be checked in all intermediate nodes between the top and the base node. Since qmp_block_stream() now uses the error from bdrv_lookup_bs()

[Qemu-block] [PATCH v4 0/8] Support streaming to an intermediate layer

New series. As I wrote in a separate e-mail, we don't need to have any additional checks on the top image before reopening it in R/W mode since bdrv_reopen_prepare() already takes care of checking the BDRV_O_ALLOW_RDWR flag, which serves our purpose. v4: - Refactor find_block_job to use the error

[Qemu-block] [PATCH 3/8] block: never cancel a streaming job without running stream_complete()

We need to call stream_complete() in order to do all the necessary clean-ups, even if there's an early failure. At the moment it's only useful to make sure that s->backing_file_str is not leaked, but it will become more important as we introduce support for streaming to any intermediate node. Sign

[Qemu-block] [PATCH 1/8] block: keep a list of block jobs

The current way to obtain the list of existing block jobs is to iterate over all root nodes and check which ones own a job. Since we want to be able to support block jobs in other nodes as well, this patch keeps a list of jobs that is updated everytime one is created or destroyed. This also updat

Re: [Qemu-block] [PATCH 4/6] block: Support streaming to an intermediate layer

On Wed 15 Apr 2015 06:09:18 PM CEST, Max Reitz wrote: >> +orig_bs_flags = bdrv_get_flags(bs); >> +if (!(orig_bs_flags & BDRV_O_RDWR)) { > > I feel like we don't want to do this if we're not streaming to an > intermediate layer but to the top layer (because that means there is > some reaso

[Qemu-block] [PATCH for-2.4 V2 2/9] block/iscsi: change all iscsilun properties from uint8_t to bool

Signed-off-by: Peter Lieven --- block/iscsi.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index be8af46..6cf7e99 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -57,9 +57,6 @@ typedef struct IscsiLun { int events; QE

[Qemu-block] [PATCH for-2.4 V2 8/9] block/iscsi: bump year in copyright notice

Signed-off-by: Peter Lieven --- block/iscsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/iscsi.c b/block/iscsi.c index 328907b..8364f97 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -2,7 +2,7 @@ * QEMU Block driver for iSCSI images * * Copyright (c) 2010-2

[Qemu-block] [PATCH for-2.4 V2 6/9] block/iscsi: increase retry count

The idea is that a command is retried in a BUSY condition up a time of approx. 60 seconds before it is failed. This should be far higher than any command timeout in the guest. Signed-off-by: Peter Lieven --- block/iscsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/

[Qemu-block] [PATCH for-2.4 V2 7/9] block/iscsi: handle SCSI_STATUS_TASK_SET_FULL

a target may issue a SCSI_STATUS_TASK_SET_FULL status if there is more than one "BUSY" command queued already. Signed-off-by: Peter Lieven --- block/iscsi.c | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index 5999f74..328907b 100644 ---

[Qemu-block] [PATCH for-2.4 V2 9/9] block/iscsi: use the allocationmap also if cache.direct=on

the allocationmap has only a hint character. The driver always double checks that blocks marked unallocated in the cache are still unallocated before taking the fast path and return zeroes. So using the allocationmap is migration safe and can also be enabled with cache.direct=on. Signed-off-by: Pe

[Qemu-block] [PATCH for-2.4 V2 5/9] block/iscsi: optimize WRITE10/16 if cache.writeback is not set

SCSI allowes to tell the target to not return from a write command if the date is not written to the disk. Use this so called FUA bit if it is supported to optimize WRITE commands if writeback is not allowed. In this case qemu always issues a WRITE followed by a FLUSH. This is 2 round trip times.

[Qemu-block] [PATCH for-2.4 V2 3/9] block/iscsi: rename iscsi_write_protected and let it return void

Signed-off-by: Peter Lieven --- block/iscsi.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index 6cf7e99..221c9fc 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -1253,11 +1253,11 @@ static void iscsi_attach_aio_context(BlockDriver

[Qemu-block] [PATCH for-2.4 V2 1/9] block/iscsi: do not forget to logout from target

We actually were always impolitely dropping the connection and not cleanly logging out. CC: qemu-sta...@nongnu.org Signed-off-by: Peter Lieven --- block/iscsi.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index ba33290..be8af46 100644 --- a/block/iscsi

[Qemu-block] [PATCH for-2.4 V2 4/9] block/iscsi: store DPOFUA bit from the modesense command

Signed-off-by: Peter Lieven --- block/iscsi.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index 221c9fc..237faa1 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -66,6 +66,7 @@ typedef struct IscsiLun { bool write_protected; bool lbpme; boo

[Qemu-block] [PATCH for-2.4 V2 0/9] various improvements for the iSCSI driver

v1->v2: - removed the requirement for libiscsi 1.10.0 [Paolo] - reworked to force_next_flush logic [Paolo] Peter Lieven (9): block/iscsi: do not forget to logout from target block/iscsi: change all iscsilun properties from uint8_t to bool block/iscsi: rename iscsi_write_protected and

Re: [Qemu-block] [PATCH for-2.4 07/10] block/iscsi: bump libiscsi requirement to 1.10.0

On 16/04/2015 14:58, Peter Lieven wrote: >> >> On 16/04/2015 14:18, Peter Lieven wrote: >>> We need this to support SCSI_STATUS_TASK_SET_FULL. >> Any reason apart from the missing constant? > > No, but I wanted to avoid starting checking for constants that were > added shortly after this. > You

Re: [Qemu-block] [PATCH for-2.4 05/10] block/iscsi: optimize WRITE10/16 if cache.writeback is not set

On 16/04/2015 15:02, Peter Lieven wrote: >>> >> Also, I think it is iscsi_co_generic_cb that should set >> force_next_flush, so that it is only set on failure. Not really for the >> optimization value, but because it's clearer. > > I don't get what you mean with it should only "set on failure".

Re: [Qemu-block] [PATCH for-2.4 05/10] block/iscsi: optimize WRITE10/16 if cache.writeback is not set

Am 16.04.2015 um 14:42 schrieb Paolo Bonzini: On 16/04/2015 14:18, Peter Lieven wrote: SCSI allowes to tell the target to not return from a write command if the date is not written to the disk. Use this so called FUA bit if it is supported to optimize WRITE commands if writeback is not allowed.

Re: [Qemu-block] [PATCH for-2.4 07/10] block/iscsi: bump libiscsi requirement to 1.10.0

Am 16.04.2015 um 14:33 schrieb Paolo Bonzini: On 16/04/2015 14:18, Peter Lieven wrote: We need this to support SCSI_STATUS_TASK_SET_FULL. Any reason apart from the missing constant? No, but I wanted to avoid starting checking for constants that were added shortly after this. You can't check

Re: [Qemu-block] [PATCH for-2.4 00/10] various improvements for the iSCSI driver

On 16/04/2015 14:18, Peter Lieven wrote: > Peter Lieven (10): > block/iscsi: do not forget to logout from target > block/iscsi: change all iscsilun properties from uint8_t to bool > block/iscsi: rename iscsi_write_protected and let it return void > block/iscsi: store DPOFUA bit from the m

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

On 16/04/2015 14:43, Kevin Wolf wrote: > > Of course cache=unsafe and cache.no-flush=on are not a good idea because > > you want to flush the qcow2 caches for example. > > Actually, cache=unsafe doesn't disable qcow2 cache writeback. That's the > difference between .bdrv_co_flush_to_os as used b

Re: [Qemu-block] [PATCH for-2.4 05/10] block/iscsi: optimize WRITE10/16 if cache.writeback is not set

On 16/04/2015 14:18, Peter Lieven wrote: > SCSI allowes to tell the target to not return from a write command > if the date is not written to the disk. Use this so called FUA > bit if it is supported to optimize WRITE commands if writeback is > not allowed. > > In this case qemu always issues a

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

Am 15.04.2015 um 11:03 hat Paolo Bonzini geschrieben: > > cachemode unsafe avoids the explicit flush which is no good idea as > > we all would agree. > > Actually, in the case of battery-backed cache file.cache.no-flush=on > (aka file.cache=unsafe) _is_ a good idea, because arrays with > battery-b

Re: [Qemu-block] [PATCH for-2.4 07/10] block/iscsi: bump libiscsi requirement to 1.10.0

On 16/04/2015 14:18, Peter Lieven wrote: > We need this to support SCSI_STATUS_TASK_SET_FULL. Any reason apart from the missing constant? Paolo > Signed-off-by: Peter Lieven > --- > configure | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/configure b/configur

Re: [Qemu-block] [Qemu-devel] [PATCH 4/6] block: Support streaming to an intermediate layer

On Thu 16 Apr 2015 02:27:39 PM CEST, Eric Blake wrote: +orig_bs_flags = bdrv_get_flags(bs); +if (!(orig_bs_flags & BDRV_O_RDWR)) { >>> >>> I feel like we don't want to do this if we're not streaming to an >>> intermediate layer but to the top layer (because that means there is >>

Re: [Qemu-block] [Qemu-devel] [PATCH 4/6] block: Support streaming to an intermediate layer

On 04/16/2015 03:36 AM, Alberto Garcia wrote: > On Wed 15 Apr 2015 06:09:18 PM CEST, Max Reitz wrote: > >>> +orig_bs_flags = bdrv_get_flags(bs); >>> +if (!(orig_bs_flags & BDRV_O_RDWR)) { >> >> I feel like we don't want to do this if we're not streaming to an >> intermediate layer but to t

[Qemu-block] [PATCH for-2.4 09/10] block/iscsi: bump year in copyright notice

Signed-off-by: Peter Lieven --- block/iscsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/iscsi.c b/block/iscsi.c index 3d0ffeb..04c1309 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -2,7 +2,7 @@ * QEMU Block driver for iSCSI images * * Copyright (c) 2010-2

[Qemu-block] [PATCH for-2.4 00/10] various improvements for the iSCSI driver

Peter Lieven (10): block/iscsi: do not forget to logout from target block/iscsi: change all iscsilun properties from uint8_t to bool block/iscsi: rename iscsi_write_protected and let it return void block/iscsi: store DPOFUA bit from the modesense command block/iscsi: optimize WRITE10/16 i

[Qemu-block] [PATCH for-2.4 07/10] block/iscsi: bump libiscsi requirement to 1.10.0

We need this to support SCSI_STATUS_TASK_SET_FULL. Signed-off-by: Peter Lieven --- configure | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/configure b/configure index 6969f6f..f73b8d0 100755 --- a/configure +++ b/configure @@ -3630,15 +3630,15 @@ if compile_prog "" ""

[Qemu-block] [PATCH for-2.4 08/10] block/iscsi: handle SCSI_STATUS_TASK_SET_FULL

a target may issue a SCSI_STATUS_TASK_SET_FULL status if there is more than one "BUSY" command queued already. Signed-off-by: Peter Lieven --- block/iscsi.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index a4902ea..3d0ffeb 100644 --- a/b

[Qemu-block] [PATCH for-2.4 10/10] block/iscsi: use the allocationmap also if cache.direct=on

the allocationmap has only a hint character. The driver always double checks that blocks marked unallocated in the cache are still unallocated before taking the fast path and return zeroes. So using the allocationmap is migration safe and can also be enabled with cache.direct=on. Signed-off-by: Pe

[Qemu-block] [PATCH for-2.4 06/10] block/iscsi: increase retry count

The idea is that a command is retried in a BUSY condition up a time of approx. 60 seconds before it is failed. This should be far higher than any command timeout in the guest. Signed-off-by: Peter Lieven --- block/iscsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/

[Qemu-block] [PATCH for-2.4 01/10] block/iscsi: do not forget to logout from target

We actually were always impolitely dropping the connection and not cleanly logging out. CC: qemu-sta...@nongnu.org Signed-off-by: Peter Lieven --- block/iscsi.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index ba33290..be8af46 100644 --- a/block/iscsi

[Qemu-block] [PATCH for-2.4 05/10] block/iscsi: optimize WRITE10/16 if cache.writeback is not set

SCSI allowes to tell the target to not return from a write command if the date is not written to the disk. Use this so called FUA bit if it is supported to optimize WRITE commands if writeback is not allowed. In this case qemu always issues a WRITE followed by a FLUSH. This is 2 round trip times.

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

Am 16.04.2015 um 13:34 schrieb Paolo Bonzini: On 16/04/2015 12:23, Peter Lieven wrote: But in this case we should be allowed to move the restriction of using the allocation map only if cache.direct = no or shouldn't we? Now that I understood the code better, probably we can remove the restrict

[Qemu-block] [PATCH for-2.4 03/10] block/iscsi: rename iscsi_write_protected and let it return void

Signed-off-by: Peter Lieven --- block/iscsi.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index 6cf7e99..221c9fc 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -1253,11 +1253,11 @@ static void iscsi_attach_aio_context(BlockDriver

[Qemu-block] [PATCH for-2.4 02/10] block/iscsi: change all iscsilun properties from uint8_t to bool

Signed-off-by: Peter Lieven --- block/iscsi.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/block/iscsi.c b/block/iscsi.c index be8af46..6cf7e99 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -57,9 +57,6 @@ typedef struct IscsiLun { int events; QE

[Qemu-block] [PATCH for-2.4 04/10] block/iscsi: store DPOFUA bit from the modesense command

Signed-off-by: Peter Lieven --- block/iscsi.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/iscsi.c b/block/iscsi.c index 221c9fc..237faa1 100644 --- a/block/iscsi.c +++ b/block/iscsi.c @@ -66,6 +66,7 @@ typedef struct IscsiLun { bool write_protected; bool lbpme; boo

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

On 16/04/2015 12:23, Peter Lieven wrote: > But in this case we should be allowed to move the restriction of > using the allocation map only if cache.direct = no or shouldn't we? Now that I understood the code better, probably we can remove the restriction, yes. > In this case cache.direct = (dr

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

Am 16.04.2015 um 11:59 schrieb Paolo Bonzini: On 16/04/2015 11:54, Peter Lieven wrote: That allocation cache in the iSCSI driver is only a hint. It always confirms blocks are really unallocated before taking the fast path returning zeroes. So I don't think it is necessary to add invalidate cach

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

On 16/04/2015 11:54, Peter Lieven wrote: > That allocation cache in the iSCSI driver is only a hint. It always > confirms blocks > are really unallocated before taking the fast path returning zeroes. > So I don't think it is necessary to add invalidate cache, or is it? > Or would you vote for rem

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

Am 16.04.2015 um 11:37 schrieb Paolo Bonzini: On 16/04/2015 09:16, Peter Lieven wrote: Thanks for the clarification. At first glance its not that obvious. There is one thing I want to mention none is not exactly writeback and directsync not exactly writethrough. We do not enable the iscsi->all

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

On 16/04/2015 09:16, Peter Lieven wrote: > Thanks for the clarification. At first glance its not that obvious. > > There is one thing I want to mention none is not exactly writeback > and directsync not exactly writethrough. We do not enable the > iscsi->allocationmap if BRDV_O_NOCACHE is in the

Re: [Qemu-block] [PATCH 4/6] block: Support streaming to an intermediate layer

On Wed 15 Apr 2015 06:09:18 PM CEST, Max Reitz wrote: >> +orig_bs_flags = bdrv_get_flags(bs); >> +if (!(orig_bs_flags & BDRV_O_RDWR)) { > > I feel like we don't want to do this if we're not streaming to an > intermediate layer but to the top layer (because that means there is > some reason

[Qemu-block] qemu exit with pending I/O

Hi, I just run tests with the new libiscsi branch that allows async reconnects. I found that I cannot quit qemu while qemu is waiting for a reconnect to the storage. E.g. I start qemu and then shut down the storage. Qemu will then loop forever in bdrv_flush() and/or bdrv_drain_all(). It would b

Re: [Qemu-block] [PATCH 2/6] block: allow block jobs in any arbitrary node

On Wed 15 Apr 2015 05:22:13 PM CEST, Max Reitz wrote: >> diff --git a/block/mirror.c b/block/mirror.c >> index 4056164..189e8f8 100644 >> --- a/block/mirror.c >> +++ b/block/mirror.c >> @@ -607,8 +607,9 @@ static void mirror_complete(BlockJob *job, Error **errp) >> return; >> } >>

Re: [Qemu-block] RFC iscsi: set FUA and DPO if !bs->enable_write_cache

Am 15.04.2015 um 11:03 schrieb Paolo Bonzini: On 14/04/2015 21:55, Peter Lieven wrote: Am 14.04.2015 um 18:15 schrieb Paolo Bonzini: On 14/04/2015 08:49, Peter Lieven wrote: Hi, Ronnie came up with an idea to reduce latency if !bs->enable_write_cache for an iSCSI device. If !bs->enable_writ