Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 02:01:59PM +0200, Wouter Verhelst wrote: > Yes. There was some discussion on that part, and we decided that setting > the flag doesn't hurt, but the spec also clarifies that using it on READ > does nothing, semantically. > > > The problem is that there are clients in the

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 13:18, Christoph Hellwig wrote: > > Yes, please do that. A "barrier" implies draining of the queue. Done -- Alex Bligh -- ___

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote: > That's probably right in the case of file-based back ends that > are running on a Linux OS. But gonbdserver for instance supports > (e.g.) Ceph based backends, where each connection might be talking > to a completely separate ceph node,

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 13:23, Christoph Hellwig wrote: > > On Thu, Sep 15, 2016 at 02:21:20PM +0200, Wouter Verhelst wrote: >> Right. So do I understand you correctly that blk-mq currently doesn't >> look at multiple queues, and just assumes that if a FLUSH is sent over >> any

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 02:21:20PM +0200, Wouter Verhelst wrote: > Right. So do I understand you correctly that blk-mq currently doesn't > look at multiple queues, and just assumes that if a FLUSH is sent over > any one of the queues, it applies to all queues? Yes. The same is true at the

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On 09/15/2016 09:17 AM, Wouter Verhelst wrote: > On Thu, Sep 15, 2016 at 01:44:29PM +0100, Alex Bligh wrote: >> >>> On 15 Sep 2016, at 13:41, Christoph Hellwig wrote: >>> >>> On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote: That's probably right in the case of

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On 15/09/2016 15:34, Eric Blake wrote: > On 09/15/2016 06:09 AM, Alex Bligh wrote: >> >> I also wonder whether any servers that can do caching per >> connection will always share a consistent cache between >> connections. The one I'm worried about in particular here >> is qemu-nbd - Eric Blake

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 01:33:20PM +0100, Alex Bligh wrote: > At an implementation level that is going to be a little difficult > for some NBD servers, e.g. ones that fork() a different process per > connection. There is in general no IPC to speak of between server > instances. Such servers would

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 13:36, Christoph Hellwig wrote: > > On Thu, Sep 15, 2016 at 01:33:20PM +0100, Alex Bligh wrote: >> At an implementation level that is going to be a little difficult >> for some NBD servers, e.g. ones that fork() a different process per >> connection.

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 13:41, Christoph Hellwig wrote: > > On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote: >> That's probably right in the case of file-based back ends that >> are running on a Linux OS. But gonbdserver for instance supports >> (e.g.) Ceph based

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 01:44:29PM +0100, Alex Bligh wrote: > > > On 15 Sep 2016, at 13:41, Christoph Hellwig wrote: > > > > On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote: > >> That's probably right in the case of file-based back ends that > >> are running on a

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On 09/15/2016 06:09 AM, Alex Bligh wrote: > > I also wonder whether any servers that can do caching per > connection will always share a consistent cache between > connections. The one I'm worried about in particular here > is qemu-nbd - Eric Blake CC'd. > I doubt that qemu-nbd would ever want

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Wouter, > The server can always refuse to allow multiple connections. Sure, but it would be neater to warn the client of that at negotiation stage (it would only be one flag, e.g. 'multiple connections unsafe'). That way the connection won't fail with a cryptic EBUSY or whatever, but will just

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 05:08:21PM +0100, Alex Bligh wrote: > Wouter, > > > The server can always refuse to allow multiple connections. > > Sure, but it would be neater to warn the client of that at negotiation > stage (it would only be one flag, e.g. 'multiple connections > unsafe'). I

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 12:09:28PM +0100, Alex Bligh wrote: > A more general point is that with multiple queues requests > may be processed in a different order even by those servers that > currently process the requests in strict order, or in something > similar to strict order. The server is

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 12:43:35PM +0100, Alex Bligh wrote: > Sure, it's at: > > https://github.com/yoe/nbd/blob/master/doc/proto.md#ordering-of-messages-and-writes > > and that link takes you to the specific section. > > The treatment of FLUSH and FUA is meant to mirror exactly the > linux

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 04:38:07AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 12:49:35PM +0200, Wouter Verhelst wrote: > > A while back, we spent quite some time defining the semantics of the > > various commands in the face of the NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA > > write

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Christoph, > It's not a write barrier - a write barrier was command that ensured that > > a) all previous writes were completed to the host/client > b) all previous writes were on non-volatile storage > > and > > c) the actual write with the barrier bit was on non-volatile storage Ah! the bit

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Paolo, > On 15 Sep 2016, at 15:07, Paolo Bonzini wrote: > > I don't think QEMU forbids multiple clients to the single server, and > guarantees consistency as long as there is no overlap between writes and > reads. These are the same guarantees you have for multiple

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Wouter, > On 15 Sep 2016, at 17:27, Wouter Verhelst wrote: > > On Thu, Sep 15, 2016 at 05:08:21PM +0100, Alex Bligh wrote: >> Wouter, >> >>> The server can always refuse to allow multiple connections. >> >> Sure, but it would be neater to warn the client of that at negotiation

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Eric, > I doubt that qemu-nbd would ever want to support the situation with more > than one client connection writing to the same image at the same time; > the implications of sorting out data consistency between multiple > writers is rather complex and not worth coding into qemu. So I think >

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On 09/15/2016 11:27 AM, Wouter Verhelst wrote: > On Thu, Sep 15, 2016 at 05:08:21PM +0100, Alex Bligh wrote: >> Wouter, >> >>> The server can always refuse to allow multiple connections. >> >> Sure, but it would be neater to warn the client of that at negotiation >> stage (it would only be one

Re: [Nbd] [PATCH 0/5] nbd improvements

Hi Jens, On 2016 M09 8, Thu 14:13:46 CEST Jens Axboe wrote: > On 09/08/2016 01:33 PM, Josef Bacik wrote: > > This is a patch series aimed at bringing NBD into 2016. The two big > > components of this series is converting nbd over to using blkmq and > > then allowing us to provide more than one

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On 15/09/2016 17:23, Alex Bligh wrote: > Paolo, > >> On 15 Sep 2016, at 15:07, Paolo Bonzini wrote: >> >> I don't think QEMU forbids multiple clients to the single server, and >> guarantees consistency as long as there is no overlap between writes and >> reads. These are

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Wouter, Josef, (& Eric) > On 15 Sep 2016, at 11:49, Wouter Verhelst wrote: > > Hi, > > On Fri, Sep 09, 2016 at 10:02:03PM +0200, Wouter Verhelst wrote: >> I see some practical problems with this: > [...] > > One more that I didn't think about earlier: > > A while back, we spent

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 12:09:28PM +0100, Alex Bligh wrote: > Wouter, Josef, (& Eric) > > > On 15 Sep 2016, at 11:49, Wouter Verhelst wrote: > > > > Hi, > > > > On Fri, Sep 09, 2016 at 10:02:03PM +0200, Wouter Verhelst wrote: > >> I see some practical problems with this: > >

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 01:55:14PM +0200, Wouter Verhelst wrote: > Maybe I'm not using the correct terminology here. The point is that > after a FLUSH, the server asserts that all write commands *for which a > reply has already been sent to the client* will also have reached > permanent storage.

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 04:52:17AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 12:46:07PM +0100, Alex Bligh wrote: > > Essentially NBD does supports FLUSH/FUA like this: > > > > https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt > > > > IE supports the same

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 12:46:07PM +0100, Alex Bligh wrote: > Essentially NBD does supports FLUSH/FUA like this: > > https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt > > IE supports the same FLUSH/FUA primitives as other block drivers (AIUI). > > Link to protocol (per

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 12:49:35PM +0200, Wouter Verhelst wrote: > A while back, we spent quite some time defining the semantics of the > various commands in the face of the NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA > write barriers. At the time, we decided that it would be unreasonable > to expect

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

On Thu, Sep 15, 2016 at 01:11:24PM +0100, Alex Bligh wrote: > > NBD_CMD_FLUSH (3) > > > > A flush request; a write barrier. > > I can see that's potentially confusing as isn't meant to mean 'an old-style > linux kernel block device write barrier'. I think in general terms it > probably is some

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

Hi, On Fri, Sep 09, 2016 at 10:02:03PM +0200, Wouter Verhelst wrote: > I see some practical problems with this: [...] One more that I didn't think about earlier: A while back, we spent quite some time defining the semantics of the various commands in the face of the NBD_CMD_FLUSH and

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 12:46, Christoph Hellwig wrote: > > On Thu, Sep 15, 2016 at 12:43:35PM +0100, Alex Bligh wrote: >> Sure, it's at: >> >> https://github.com/yoe/nbd/blob/master/doc/proto.md#ordering-of-messages-and-writes >> >> and that link takes you to the specific

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

> On 15 Sep 2016, at 12:52, Christoph Hellwig wrote: > > On Thu, Sep 15, 2016 at 12:46:07PM +0100, Alex Bligh wrote: >> Essentially NBD does supports FLUSH/FUA like this: >> >> https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt >> >> IE supports the