Re: [Nbd] SUMMARY: Re: [Qemu-devel] [RFC 1/1] nbd (specification): add NBD_CMD_WRITE_ZEROES command

2016-02-19 Thread Vladimir Sementsov-Ogievskiy
On 19.02.2016 10:12, Denis V. Lunev wrote: > On 02/18/2016 08:23 PM, Denis V. Lunev wrote: >> On 02/18/2016 07:35 PM, Eric Blake wrote: >>> On 02/18/2016 02:18 AM, Roman Kagan wrote: On Wed, Feb 17, 2016 at 01:58:47PM -0700, Eric Blake wrote: > On 02/17/2016 11:10 AM, Denis V. Lunev wrote:

Re: [Nbd] [Qemu-devel] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-07 Thread Vladimir Sementsov-Ogievskiy
On 05.04.2016 16:43, Paolo Bonzini wrote: > > On 05/04/2016 06:05, Kevin Wolf wrote: >> The options I can think of is adding a request field "max number of >> descriptors" or a flag "only single descriptor" (with the assumption >> that clients always want one or unlimited), but maybe you have a bet

[Nbd] write_zeroes/trim on the whole disk

2016-09-23 Thread Vladimir Sementsov-Ogievskiy
Hi all! There is a following problem. When we need to write_zeroes or trim the whole disk, we have to do it iteratively, because of 32-bit restriction on request length. For example, current implementation of mirror (see mirror_dirty_init()) do this by chunks of 2147418112 bytes (with default

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 00:21, Wouter Verhelst wrote: > On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: >> My preference would be a new flag to the existing commands, with >> explicit documentation that 0 offset and 0 length must be used with that >> flag, when requesting a full-device wipe. > Al

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 15:06, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 00:21, Wouter Verhelst wrote: >> On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: >>> My preference would be a new flag to the existing commands, with >>> explicit documentation that 0

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 15:06, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 00:21, Wouter Verhelst wrote: >> On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: >>> My preference would be a new flag to the existing commands, with >>> explicit documentation that 0

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 16:42, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 15:06, Vladimir Sementsov-Ogievskiy wrote: >> On 24.09.2016 00:21, Wouter Verhelst wrote: >>> On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: >>>> My preference would be a new flag

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 19:31, Alex Bligh wrote: >> On 24 Sep 2016, at 13:06, Vladimir Sementsov-Ogievskiy >> wrote: >> >> Note: if disk size is not aligned to X we will have to send request larger >> than the disk size to clear the whole disk. > If you look at the bloc

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 19:35, Alex Bligh wrote: >> On 24 Sep 2016, at 17:20, Vladimir Sementsov-Ogievskiy >> wrote: >> >> Also, accordingly to documentation, NBD_CMD_TRIM is not appropriate for disk >> clearing: >> >> * `NBD_CMD_TRIM` (4) >> >>

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 19:44, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 19:35, Alex Bligh wrote: >>> On 24 Sep 2016, at 17:20, Vladimir Sementsov-Ogievskiy >>> wrote: >>> >>> Also, accordingly to documentation, NBD_CMD_TRIM is not appropriate >>&

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 19:49, Alex Bligh wrote: >> On 24 Sep 2016, at 17:42, Vladimir Sementsov-Ogievskiy >> wrote: >> >> On 24.09.2016 19:31, Alex Bligh wrote: >>>> On 24 Sep 2016, at 13:06, Vladimir Sementsov-Ogievskiy >>>> wrote: >>>> >

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 20:13, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 19:49, Alex Bligh wrote: >>> On 24 Sep 2016, at 17:42, Vladimir Sementsov-Ogievskiy >>> wrote: >>> >>> On 24.09.2016 19:31, Alex Bligh wrote: >>>>> On 24 Sep 2016

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 20:32, Alex Bligh wrote: >> On 24 Sep 2016, at 18:13, Vladimir Sementsov-Ogievskiy >> wrote: >> >> On 24.09.2016 19:49, Alex Bligh wrote: >>>> On 24 Sep 2016, at 17:42, Vladimir Sementsov-Ogievskiy >>>> wrote: >>>> >&

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 21:24, Alex Bligh wrote: >> On 24 Sep 2016, at 18:47, Vladimir Sementsov-Ogievskiy >> wrote: >> >> I just wanted to say, that if we want a possibility of clearing the whole >> disk in one request for qcow2 we have to take 512 as granularity for such

Re: [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Vladimir Sementsov-Ogievskiy
On 24.09.2016 23:14, Carl-Daniel Hailfinger wrote: > On 24.09.2016 19:33, Vladimir Sementsov-Ogievskiy wrote: >> On 24.09.2016 20:13, Vladimir Sementsov-Ogievskiy wrote: >>> I agree that requests larger than disk size are ugly.. But splitting >>> request brings me again

[Nbd] [PATCH] proto: add 'shift' extension.

2016-09-26 Thread Vladimir Sementsov-Ogievskiy
This extension allows big requests for TRIM and WRITE_ZEROES through special 'shift' parameter, which means that offset and length should be shifted left by several bits. This is needed for efficient clearing large regions of the disk (up to the whole disk). Signed-off-by: Vladimir

Re: [Nbd] [PATCH] proto: add 'shift' extension.

2016-09-26 Thread Vladimir Sementsov-Ogievskiy
On 26.09.2016 15:51, Paolo Bonzini wrote: > > On 26/09/2016 14:46, Vladimir Sementsov-Ogievskiy wrote: >> This extension allows big requests for TRIM and WRITE_ZEROES through >> special 'shift' parameter, which means that offset and length should be >> shifted l

Re: [Nbd] [PATCH] proto: add 'shift' extension.

2016-09-28 Thread Vladimir Sementsov-Ogievskiy
gt; On 27/09/2016 15:28, Denis V. Lunev wrote: >>>>>> On 09/27/2016 03:07 PM, Paolo Bonzini wrote: >>>>>>> - Original Message - >>>>>>>> From: "Denis V. Lunev" >>>>>>>> To: "Paolo Bonzini&qu

[Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-25 Thread Vladimir Sementsov-Ogievskiy
CC: Wouter Verhelst CC: Paolo Bonzini CC: Kevin Wolf CC: Stefan Hajnoczi Signed-off-by: Eric Blake Signed-off-by: Vladimir Sementsov-Ogievskiy --- v3: Hi all. This is almost a resend of v2 (by Eric Blake), The only change is removing the restriction, that sum of status descriptor lengths must b

Re: [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Vladimir Sementsov-Ogievskiy
29.11.2016 13:18, Kevin Wolf wrote: > Am 27.11.2016 um 20:17 hat Wouter Verhelst geschrieben: >>> 3. Q: selecting of dirty bitmap to export >>> A: several variants: >>>1: id of bitmap is in flags field of request >>>pros: - simple >>>cons: - it's a hack. flags fi

Re: [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Vladimir Sementsov-Ogievskiy
Hi, 29.11.2016 13:50, Wouter Verhelst wrote: Hi, On Mon, Nov 28, 2016 at 06:33:24PM +0100, Wouter Verhelst wrote: However, I'm arguing that if we're going to provide information about snapshots, we should be able to properly refer to these snapshots from within an NBD context. My previous mail

Re: [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Vladimir Sementsov-Ogievskiy
29.11.2016 15:57, Alex Bligh wrote: > Vladimir, > > I went back to April to reread the previous train of conversation > then found you had helpfully summarised some if it. Comments > below. > > Rather than comment on many of the points individual, the root > of my confusion and to some extent uncom

Re: [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Vladimir Sementsov-Ogievskiy
29.11.2016 17:52, Alex Bligh wrote: > Vladimir, > 4. Q: Should not get_{allocated,dirty} be separate commands? cons: Two commands with almost same semantic and similar means? pros: However here is a good point of separating clearly defined and native for blo

Re: [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-01 Thread Vladimir Sementsov-Ogievskiy
01.12.2016 13:14, Wouter Verhelst wrote: > Here's another update. > > Changes since previous version: > - Rename "allocation context" to "metadata context" > - Stop making metadata context 0 be special; instead, name it >"BASE:allocation" and allow it to be selected like all other contexts. > -

Re: [Nbd] [Qemu-devel] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-02 Thread Vladimir Sementsov-Ogievskiy
02.12.2016 02:42, John Snow wrote: > (B) In the case of "User just wanted to look around," the bitmap should > be merged back into the bitmap it was forked from. currently existing example: "failed incremental backup" -- Best regards, Vladimir --

Re: [Nbd] [Qemu-devel] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-05 Thread Vladimir Sementsov-Ogievskiy
02.12.2016 23:39, John Snow wrote: On 12/02/2016 01:45 PM, Alex Bligh wrote: John, +Some storage formats and operations over such formats express a +concept of data dirtiness. Whether the operation is block device +mirroring, incremental block device backup or any other operation with +a conc

Re: [Nbd] [Qemu-devel] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-07 Thread Vladimir Sementsov-Ogievskiy
08.12.2016 06:39, Alex Bligh wrote: [...] >> are vendor dependent extensions. The 'BASE' namespace represents >> metadata contexts defined within this document. Other namespaces >> may be registered by reference within this document. >> >> +The "BASE:allocation" >> >> Backticks: `BASE:allocation`

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 18:08, Alex Bligh wrote: > (NB: I've already applied this and pushed it) > > * Change NBD_OPT_LIST_METADATA etc. to explicitly send a list of queries >and add a count of queries so we can extend the command later (rather than >rely on the length of option) > > * For NBD_OPT_LIST_

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 19:38, Alex Bligh wrote: Vladimir, +non-zero number of metadata contexts during negotiation. Servers SHOULD +reply to clients sending `NBD_CMD_BLOCK_STATUS without backquote Fixed +If zero queries are sent, then the server MUST return all +the metadata contexts it knows a

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 20:09, Wouter Verhelst wrote: > On Wed, Dec 14, 2016 at 03:08:40PM +, Alex Bligh wrote: >> (NB: I've already applied this and pushed it) > Thanks. > >> * Change NBD_OPT_LIST_METADATA etc. to explicitly send a list of queries >>and add a count of queries so we can extend the comma

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 20:36, Alex Bligh wrote: >> On 14 Dec 2016, at 17:05, Vladimir Sementsov-Ogievskiy >> wrote: >>> However, this raises another question. Wouter deliberately made the >>> query format freeform so that you could e.g. set a context like: >>&

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-26 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 21:17, Alex Bligh wrote: On 14 Dec 2016, at 17:55, Vladimir Sementsov-Ogievskiy wrote: Hmmm... Well in the '*' case, it's up to the namespace as to how it parses things, so '*' could be prohibited entirely or mean 'return the first 20 matching&#x

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-26 Thread Vladimir Sementsov-Ogievskiy
14.12.2016 22:01, Alex Bligh wrote: > Wouter, > > (Our mails crossed and I've actually pushed something, but no matter) > >> On 14 Dec 2016, at 18:49, Wouter Verhelst wrote: >> >> What I was trying to say is that I think the result to _LIST_ with no >> queries should return all information the cli

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-27 Thread Vladimir Sementsov-Ogievskiy
A bit out of topic, but... > structured replies via `NBD_OPT_STRUCTURED_REPLY`. Conversely, if > structured replies are negotiated, the server MUST use a > structured reply for any response with a payload, and MUST NOT use > a simple reply for `NBD_CMD_READ` (even for the case of an early > `EINV

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-29 Thread Vladimir Sementsov-Ogievskiy
29.12.2016 19:04, Alex Bligh wrote: >> On 28 Dec 2016, at 00:18, Wouter Verhelst wrote: >> >> On Mon, Dec 26, 2016 at 05:52:54PM +0300, Vladimir Sementsov-Ogievskiy wrote: >>> Shouldn't we add some flags to REP_META_CONTEXT, for client to be insure, is >>>

Re: [Nbd] [PATCH] Further tidy-up on block status

2016-12-29 Thread Vladimir Sementsov-Ogievskiy
29.12.2016 19:08, Alex Bligh wrote: > Vladimir, > >> On 26 Dec 2016, at 15:57, Vladimir Sementsov-Ogievskiy >> wrote: >> >>> OK, so first of all, one of the changes I made earlier was that now >>> each of the commands carries a list of queries, the wa

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-11 Thread Vladimir Sementsov-Ogievskiy
from current version: >>> If an error occurs, the server SHOULD set the appropriate error code in the error field of an error chunk. However, if the error does not involve invalid usage (such as a request beyond the bounds of the file), a server MAY reply with a single block status descriptor

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-11 Thread Vladimir Sementsov-Ogievskiy
11.01.2017 22:00, Alex Bligh wrote: >> On 11 Jan 2017, at 15:31, Vladimir Sementsov-Ogievskiy >> wrote: >> >>>>> If an error occurs, the server SHOULD set the appropriate error code in >>>>> the error field of an error chunk. However, if the erro

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-12 Thread Vladimir Sementsov-Ogievskiy
(Sorry, it may be a bit out of topic, but I hope it is comfortable for all that I just comment current version of the feature by answering cover letter of last related patch set) From |"NBD_OPT_LIST_META_CONTEXT| (9)": The server MUST either reply with an error (for instance |EINVAL| if the op

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-13 Thread Vladimir Sementsov-Ogievskiy
12.01.2017 16:11, Alex Bligh wrote: >> On 12 Jan 2017, at 07:05, Vladimir Sementsov-Ogievskiy >> wrote: >> >> Yes this is better. But is it actually needed to force contexts have some >> safe default? If context wants it may define such default without this >

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-16 Thread Vladimir Sementsov-Ogievskiy
13.01.2017 13:29, Alex Bligh wrote: >> On 13 Jan 2017, at 09:48, Vladimir Sementsov-Ogievskiy >> wrote: >> >> 12.01.2017 16:11, Alex Bligh wrote: >>>> On 12 Jan 2017, at 07:05, Vladimir Sementsov-Ogievskiy >>>> wrote: >>>> >>

Re: [Nbd] [PATCH] Further tidy-up on block status

2017-01-20 Thread Vladimir Sementsov-Ogievskiy
About extents: is 32bit length enough? We will have to send 4096 for empty 16tb disk.. -- Best regards, Vladimir -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! ht

[Nbd] nbd structured reply

2017-09-21 Thread Vladimir Sementsov-Ogievskiy
Hi all! I'm about this: "A server SHOULD try to minimize the number of chunks sent in a reply, but MUST NOT mark a chunk as final if there is still a possibility of detecting an error before transmission of that chunk completes" What do we mean by "possibility"? Formally, such possibility ex

[Nbd] nbd structured reply

2017-09-21 Thread Vladimir Sementsov-Ogievskiy via Nbd-general
Hi all! I'm about this: "A server SHOULD try to minimize the number of chunks sent in a reply, but MUST NOT mark a chunk as final if there is still a possibility of detecting an error before transmission of that chunk completes" What do we mean by "possibility"? Formally, such possibility ex