For ease of management it would be nice for users to specify that the
device node for a nbd device is destroyed once it is disconnected and
there are no more users. Add a client flag and enable this operation to
happen.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block
for less than the dead connection timeout time and bring it
back up and everything resumes gracefully.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 63 +---
include/uapi/linux/nbd-netlink.h | 1 +
2 files changed, 60 inse
Allow users to query the status of existing nbd devices. Right now this
only returns whether or not the device is connected, but could be
extended in the future to include more information.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
This patchset adds a new netlink configuration interface to NBD as well as a
bunch of enhancments around path failures. The patches provide the following
enhancemnts to NBD
- Netlink configuration interface that doesn't leave a userspace application
waiting in kernel space for the device to
than are preallocated at insmod time. We also have gained the ability
to not specify a particular device and be provided one for us so that
userspace doesn't need to find a free device to configure.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
In preparation for the upcoming netlink interface we need to not rely on
already having the bdev for the NBD device we are doing operations on.
Instead of passing the bdev around, just use it in places where we know
we already have the bdev.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
d
to configure the device. This has
a nice side-effect of fixing a long standing problem where you could end
up with a half-configured nbd device that needed to be "disconnected" in
order to be usable again. Now once we close our device the
configuration will be discarded.
Signed-off-by: Josef
Provide a mechanism to notify userspace that there's been a link problem
on a NBD device. This will allow userspace to re-establish a connection
and provide the new socket to the device without disrupting the device.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block
When adding a new socket we look it up and then try to add it to our
configuration. If any of those steps fail we need to make sure we put
the socket so we don't leak them.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 9 +++--
1 file changed, 7 insertions
the requests. So instead make the ioctl only shutdown
the sockets and make it so that we only ever run nbd_clear_que from the
device teardown.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/block/n
one we mark it
as dead and send all IO destined for that socket to one of the other healthy
sockets. Any outstanding requests that were on the dead socket will timeout and
be re-submitted properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
ck/for-next, so the latter might require
> a respin.
Tested with nbd, didn't crash, you can add
Tested-by: Josef Bacik <jba...@fb.com>
Thanks,
Josef
to be writeable allows us to be able to control our io limits more
precisely.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
block/blk-sysfs.c | 16 +++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 4585426..e7db3f8 100644
--- a/blo
From: Josef Bacik <jba...@fb.com>
We've been relying on the block layer to assume rq->errors being set
translates into -EIO. I noticed in testing that sometimes this isn't
true, and really there's not much of a reason to have a counter instead
of just using -EIO. So set it properly so
ped state by ext4_commit_super().
[mlin: ported to 4.11-rc2]
Signed-off-by: Ratna Manoj Bolla <manoj...@gmail.com
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/bl
blk_mq_run_hw_queue(hctx, true);
> }
>
> - return queued != 0;
> + return (queued + errors) != 0;
> }
>
> static void __blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx)
>
Thanks this fixed it, you can add
Tested-by: Josef Bacik <jo...@toxicpanda.com>
Thanks,
Josef
Yeah I think this is ok, I’ll throw it on my queue for fixes for this cycle.
Thanks,
Josef
On 3/20/17, 6:58 PM, "Ming Lin" wrote:
From: Ratna Manoj Bolla
When a filesystem is mounted on a nbd device and on a disconnect, because
of kill_bdev(), and
On Wed, 2017-03-08 at 11:07 +0100, Wouter Verhelst wrote:
> On Tue, Feb 28, 2017 at 11:57:11AM -0500, Josef Bacik wrote:
> >
> > The existing ioctl interface for configuring NBD devices is a bit
> > cumbersome and hard to extend. The other problem is we leave a
> > us
We can't just set the timeout on the tagset, we have to set it on the
queue as it would have been setup already at this point.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/block/nbd.c b/d
we only allow a device to be configured if it's
ref count is currently 0.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 210 +---
1 file changed, 133 insertions(+), 77 deletions(-)
diff --git a/drivers/block/nbd.c b/d
In preparation for the upcoming netlink interface we need to not rely on
already having the bdev for the NBD device we are doing operations on.
Instead of passing the bdev around, just use it in places where we know
we already have the bdev.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
d
This is kind of a big batch of patches, but they all depend on eachother so it
was hard to tease out the fixes from the enhancements without making my life
miserable.
FIXES:
nbd: set queue timeout properly
nbd: handle ERESTARTSYS properly
The ERSTARTSYS one in particular is pretty awful as we
one we mark it
as dead and send all IO destined for that socket to one of the other healthy
sockets. Any outstanding requests that were on the dead socket will timeout and
be re-submitted properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
If we fail to register the blockdev we need to make sure to destroy the
recv workqueue.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 0623f8f..a
-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 88 ++---
1 file changed, 50 insertions(+), 38 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 25891a1..0623f8f 100644
--- a/drivers/block/nbd.c
+++ b/d
On Fri, 2017-02-10 at 21:07 +0100, Alex Bligh wrote:
> >
> > On 10 Feb 2017, at 19:06, Josef Bacik <jba...@fb.com> wrote:
> >
> > We noticed when trying to do O_DIRECT to an export on the server
> > side
> > that we were getting requests smalle
Calling blk_queue_make_request resets a bunch of settings on the
request_queue, but all we really want is to update the make_request_fn,
so do this directly so we don't lose things like the logical and
physical block sizes.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
block/blk-mq
blocksizes and then calling bd_set_size.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 39 ---
1 file changed, 16 insertions(+), 23 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 34a280a..e0d770c
.
We also need to make sure that somebody calling SET_SOCK isn't coming in
before we're done with our socks array, so add a waitqueue to wait on
previous users of the socks array before initiating a new socks array.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
Break the ioctl handling out into helper functions, some of these things
are getting pretty big and unwieldy.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 265 ++--
1 file changed, 130 insertions(+), 135 deletions(-)
On Mon, 2017-02-06 at 10:54 -0500, Josef Bacik wrote:
> The way we make changes to the NBD device is inherently racey, as we
> could be in the middle of a request and suddenly change the number of
> connections. In practice this isn't a big deal, but with timeouts we
>
values don't change. Fix this by freezing the queue
before we do any of our device operations.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 1
one we mark it
as dead and send all IO destined for that socket to one of the other healthy
sockets. Any outstanding requests that were on the dead socket will timeout and
be re-submitted properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
To prepare for dynamically adding new nbd devices to the system switch
from using an array for the nbd devices and instead use an idr. This
copies what loop does for keeping track of its devices.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
Since we are in the memory reclaim path we need our recv work to be on a
workqueue that has WQ_MEM_RECLAIM set so we can avoid deadlocks. Also
set WQ_HIGHPRI since we are in the completion path for IO.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 13 +++--
On Tue, Jan 24, 2017 at 2:11 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
On Mon, Jan 23, 2017 at 10:52:42AM -0500, Josef Bacik wrote:
On Mon, Jan 23, 2017 at 10:03 AM, Greg KH
<gre...@linuxfoundation.org>
wrote:
> On Mon, Jan 23, 2017 at 09:57:52AM -0500, Josef Bacik wrot
On Mon, Jan 23, 2017 at 10:03 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
On Mon, Jan 23, 2017 at 09:57:52AM -0500, Josef Bacik wrote:
On Mon, Jan 23, 2017 at 9:52 AM, Greg KH
<gre...@linuxfoundation.org> wrote:
> On Mon, Jan 23, 2017 at 09:42:08AM -0500, Jo
On Mon, Jan 23, 2017 at 9:52 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
On Mon, Jan 23, 2017 at 09:42:08AM -0500, Josef Bacik wrote:
On Sat, Jan 21, 2017 at 4:05 AM, Greg KH
<gre...@linuxfoundation.org> wrote:
> On Fri, Jan 20, 2017 at 04:56:52PM -0500, Jo
> On Jan 21, 2017, at 4:03 AM, Greg KH <gre...@linuxfoundation.org> wrote:
>
>> On Fri, Jan 20, 2017 at 04:56:50PM -0500, Josef Bacik wrote:
>> NBD is moving to a loop-control esque way of managing it's device
>> creation and deletion. Reserver a minor number for
> On Jan 21, 2017, at 7:12 AM, Wouter Verhelst <w...@uter.be> wrote:
>
>> On Fri, Jan 20, 2017 at 04:56:52PM -0500, Josef Bacik wrote:
>> This patch mirrors the loop back device behavior with a few changes. First
>> there is no DEL operation as NBD doesn't get
devices as it
causes problems for them in containers. However allow this to be optional as
things like the OSS NBD client probably doesn't care and would like to just be
given a device to use.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
To prepare for dynamically adding new nbd devices to the system switch
from using an array for the nbd devices and instead use an idr. This
copies what loop does for keeping track of its devices.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
NBD is moving to a loop-control esque way of managing it's device
creation and deletion. Reserver a minor number for the new device.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
include/linux/miscdevice.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/linux/miscdevice.h b/i
Since we are in the memory reclaim path we need our recv work to be on a
workqueue that has WQ_MEM_RECLAIM set so we can avoid deadlocks. Also
set WQ_HIGHPRI since we are in the completion path for IO.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 13 +++--
A user noticed that write performance was horrible over loopback and we
traced it to an inversion of when we need to set MSG_MORE. It should be
set when we have more bvec's to send, not when we are on the last bvec.
This patch made the test go from 20 iops to 78k iops.
Signed-off-by: Josef Bacik
On Mon, Jan 16, 2017 at 3:29 PM, Sagi Grimberg wrote:
Hey Josef,
I'm going to use it the same way loop does, there will be a
/dev/nbd-control where you can say ADD, REMOVE, and GET_NEXT. I need
the
search functionality to see if we are adding something that already
exists,
On Sat, Jan 14, 2017 at 4:10 PM, Sagi Grimberg wrote:
Hey Josef,
To prepare for dynamically adding new nbd devices to the system
switch
from using an array for the nbd devices and instead use an idr.
This
copies what loop does for keeping track of its devices.
I think
On Sat, Jan 14, 2017 at 6:31 PM, Christoph Anton Mitterer
wrote:
Hi.
On advice from Alex Bligh I'd like to ping linux-block and nbd-general
about the issue described here:
https://github.com/NetworkBlockDevice/nbd/issues/44
What basically happens is, that with a recent
> On Jan 14, 2017, at 4:15 PM, Sagi Grimberg wrote:
>
>
>>> Hey Josef,
>>>
Since we are in the memory reclaim path we need our recv work to be on a
workqueue that has WQ_MEM_RECLAIM set so we can avoid deadlocks. Also
set WQ_HIGHPRI since we are in the
> On Jan 13, 2017, at 5:41 PM, Sagi Grimberg wrote:
>
> Hey Josef,
>
>> To prepare for dynamically adding new nbd devices to the system switch
>> from using an array for the nbd devices and instead use an idr. This
>> copies what loop does for keeping track of its devices.
>
> On Jan 13, 2017, at 5:24 PM, Sagi Grimberg wrote:
>
> Hey Josef,
>
>> Since we are in the memory reclaim path we need our recv work to be on a
>> workqueue that has WQ_MEM_RECLAIM set so we can avoid deadlocks. Also
>> set WQ_HIGHPRI since we are in the completion path for
To prepare for dynamically adding new nbd devices to the system switch
from using an array for the nbd devices and instead use an idr. This
copies what loop does for keeping track of its devices.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
Since we are in the memory reclaim path we need our recv work to be on a
workqueue that has WQ_MEM_RECLAIM set so we can avoid deadlocks. Also
set WQ_HIGHPRI since we are in the completion path for IO.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 15
The old maintainers email is bouncing and I've rewritten most of this
driver in the recent months. Also add linux-block to the mailinglist
and remove the old tree, I will send patches through the linux-block
tree. Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
MAINTAINERS | 4 +
On Mon, Dec 5, 2016 at 4:27 PM, Eric Blake <ebl...@redhat.com> wrote:
On 12/05/2016 03:20 PM, Josef Bacik wrote:
While doing stress tests we noticed that we'd get a lot of dmesg
spam if
we suddenly disconnected teh nbd device out of band. Rate limite
the
s/teh/the/
s/limite
While doing stress tests we noticed that we'd get a lot of dmesg spam if
we suddenly disconnected teh nbd device out of band. Rate limite the
messages in the io path in order to deal with this.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 23
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
V4->V5:
-Updated the multi-connection flag to be an actual unused value.
V3->V4:
-Fixed a problem where fast completes (or early completes) would crash because
we were still accessing the bio's on the submit side.
-
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
V3->V4:
-Fixed a problem where fast completes (or early completes) would crash because
we were still accessing the bio's on the submit side.
-Added a flag to disallow multi-connection support if the server doesn't
e
On 10/07/2016 02:36 PM, Pavel Machek wrote:
On Thu 2016-09-15 10:43:54, Josef Bacik wrote:
NBD can become contended on its single connection. We have to serialize all
writes and we can only process one read response at a time. Fix this by
allowing userspace to provide multiple connections
On 10/03/2016 07:34 AM, Alex Bligh wrote:
On 3 Oct 2016, at 08:57, Christoph Hellwig wrote:
Can you clarify what you mean by that? Why is it an "odd flush
definition", and how would you "properly" define it?
E.g. take the defintion from NVMe which also supports
On 09/29/2016 12:41 PM, Wouter Verhelst wrote:
On Thu, Sep 29, 2016 at 10:03:50AM -0400, Josef Bacik wrote:
So think of it like normal disks with multiple channels. We don't send flushes
down all the hwq's to make sure they are clear, we leave that decision up to the
application (usually a FS
On 09/29/2016 05:52 AM, Wouter Verhelst wrote:
Hi Josef,
On Wed, Sep 28, 2016 at 04:01:32PM -0400, Josef Bacik wrote:
NBD can become contended on its single connection. We have to serialize all
writes and we can only process one read response at a time. Fix this by
allowing userspace
> On Sep 22, 2016, at 9:44 PM, Ming Lei wrote:
>
>> On Thu, Sep 22, 2016 at 11:17 PM, Jens Axboe wrote:
>>> On 09/22/2016 09:12 AM, Christoph Hellwig wrote:
>>>
On Thu, Sep 22, 2016 at 09:03:56AM -0600, Jens Axboe wrote:
Having to grab a
mq path as well.
You can add my
Tested-by: Josef Bacik <jba...@fb.com>
to these. Thanks,
Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On 09/22/2016 10:59 AM, Christoph Hellwig wrote:
On Thu, Sep 22, 2016 at 08:53:00AM -0600, Jens Axboe wrote:
If a driver sets BLK_MQ_F_BLOCKING, it is allowed to block in its
->queue_rq() handler. For that case, blk-mq ensures that we always
calls it from a safe context.
First can you provide
On 09/15/2016 09:17 AM, Wouter Verhelst wrote:
On Thu, Sep 15, 2016 at 01:44:29PM +0100, Alex Bligh wrote:
On 15 Sep 2016, at 13:41, Christoph Hellwig wrote:
On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote:
That's probably right in the case of file-based back
On 09/10/2016 03:43 AM, Christoph Hellwig wrote:
Hi Josef,
I haven't read the full path as I'm a bit in a hurry, but is there
a good reason to not simply have a socket per-hw_ctx and store it in
the hw_ctx private data instead of using the index in the nbd_cmd
structure?
No good reason, just
We hit a warning when shutting down the nbd connection because we have irq's
disabled. We don't really need to do the shutdown under the lock, just clear
the nbd->sock. So do the shutdown outside of the irq. This gets rid of the
warning.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
Instead of rolling our own timer, just utilize the blk mq req timeout and do the
disconnect if any of our commands timeout.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 51 ++-
1 file changed, 14 insertions(+), 37 del
In preparation for some future changes, change a few of the state bools over to
normal bits to set/clear properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/block/n
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 267
1 file changed, 143 insertions(+), 124 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4c6dd1a..4aa45ed
This moves NBD over to using blkmq, which allows us to get rid of the NBD
wide queue lock and the async submit kthread. We will start with 1 hw
queue for now, but I plan to add multiple tcp connection support in the
future and we'll fix how we set the hwqueue's.
Signed-off-by: Josef Bacik <
101 - 172 of 172 matches
Mail list logo