This moves NBD over to using blkmq, which allows us to get rid of the NBD
wide queue lock and the async submit kthread. We will start with 1 hw
queue for now, but I plan to add multiple tcp connection support in the
future and we'll fix how we set the hwqueue's.
Signed-off-by: Josef Bacik <
We hit a warning when shutting down the nbd connection because we have irq's
disabled. We don't really need to do the shutdown under the lock, just clear
the nbd->sock. So do the shutdown outside of the irq. This gets rid of the
warning.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
This moves NBD over to using blkmq, which allows us to get rid of the NBD
wide queue lock and the async submit kthread. We will start with 1 hw
queue for now, but I plan to add multiple tcp connection support in the
future and we'll fix how we set the hwqueue's.
Signed-off-by: Josef Bacik <
one we mark it
as dead and send all IO destined for that socket to one of the other healthy
sockets. Any outstanding requests that were on the dead socket will timeout and
be re-submitted properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
On Mon, 2017-02-06 at 10:54 -0500, Josef Bacik wrote:
> The way we make changes to the NBD device is inherently racey, as we
> could be in the middle of a request and suddenly change the number of
> connections. In practice this isn't a big deal, but with timeouts we
>
.
We also need to make sure that somebody calling SET_SOCK isn't coming in
before we're done with our socks array, so add a waitqueue to wait on
previous users of the socks array before initiating a new socks array.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
V1->V2:
-re
.
We also need to make sure that somebody calling SET_SOCK isn't coming in
before we're done with our socks array, so add a waitqueue to wait on
previous users of the socks array before initiating a new socks array.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
Break the ioctl handling out into helper functions, some of these things
are getting pretty big and unwieldy.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 265 ++--
1 file changed, 130 insertions(+), 135 deletions(-)
values don't change. Fix this by freezing the queue
before we do any of our device operations.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 1
On Sat, Jan 21, 2017 at 4:05 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
> On Fri, Jan 20, 2017 at 04:56:52PM -0500, Josef Bacik wrote:
>> This patch mirrors the loop back device behavior with a few
>> changes. First
>> there is no DEL operation as NBD doesn'
On Mon, Jan 23, 2017 at 10:03 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
> On Mon, Jan 23, 2017 at 09:57:52AM -0500, Josef Bacik wrote:
>> On Mon, Jan 23, 2017 at 9:52 AM, Greg KH
>> <gre...@linuxfoundation.org> wrote:
>> > On Mon, Jan 23, 2017 at
On Tue, Jan 24, 2017 at 2:11 AM, Greg KH <gre...@linuxfoundation.org>
wrote:
> On Mon, Jan 23, 2017 at 10:52:42AM -0500, Josef Bacik wrote:
>> On Mon, Jan 23, 2017 at 10:03 AM, Greg KH
>> <gre...@linuxfoundation.org>
>> wrote:
>> > On Mon, Jan 23, 2
devices as it
causes problems for them in containers. However allow this to be optional as
things like the OSS NBD client probably doesn't care and would like to just be
given a device to use.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
To prepare for dynamically adding new nbd devices to the system switch
from using an array for the nbd devices and instead use an idr. This
copies what loop does for keeping track of its devices.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
blocksizes and then calling bd_set_size.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 39 ---
1 file changed, 16 insertions(+), 23 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 34a280a..e0d770c
On Fri, 2017-02-10 at 21:07 +0100, Alex Bligh wrote:
> >
> > On 10 Feb 2017, at 19:06, Josef Bacik <jba...@fb.com> wrote:
> >
> > We noticed when trying to do O_DIRECT to an export on the server
> > side
> > that we were getting requests smalle
-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 88 ++---
1 file changed, 50 insertions(+), 38 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 25891a1..0623f8f 100644
--- a/drivers/block/nbd.c
+++ b/d
On Wed, 2017-02-08 at 10:05 -0500, Josef Bacik wrote:
> In preparation for allowing seamless reconnects we need a way to make
> sure that we don't free the socks array out from underneath
> ourselves.
> So a socks_ref counter in order to keep track of who is using the
> socks
&g
If we fail to register the blockdev we need to make sure to destroy the
recv workqueue.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 0623f8f..a
and requeue the IO. If anybody tries to
submit a request that isn't our pending request then requeue that
request until we are able to service the one that is pending.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 138 +++-
than are preallocated at insmod time. We also have gained the ability
to not specify a particular device and be provided one for us so that
userspace doesn't need to find a free device to configure.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
In preparation for the upcoming netlink interface we need to not rely on
already having the bdev for the NBD device we are doing operations on.
Instead of passing the bdev around, just use it in places where we know
we already have the bdev.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
d
We can't just set the timeout on the tagset, we have to set it on the
queue as it would have been setup already at this point.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/block/nbd.c b/d
we only allow a device to be configured if it's
ref count is currently 0.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 210 +---
1 file changed, 133 insertions(+), 77 deletions(-)
diff --git a/drivers/block/nbd.c b/d
This is kind of a big batch of patches, but they all depend on eachother so it
was hard to tease out the fixes from the enhancements without making my life
miserable.
FIXES:
nbd: set queue timeout properly
nbd: handle ERESTARTSYS properly
The ERSTARTSYS one in particular is pretty awful as we
This was introduced in the multi-connection patch, we've been leaking
socket's ever since.
Fixes: 9561a7a ("nbd: add multi-connection support")
cc: sta...@vger.kernel.org
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 4 +++-
1 file changed, 3 insertion
On 09/10/2016 03:43 AM, Christoph Hellwig wrote:
> Hi Josef,
>
> I haven't read the full path as I'm a bit in a hurry, but is there
> a good reason to not simply have a socket per-hw_ctx and store it in
> the hw_ctx private data instead of using the index in the nbd_cmd
> structure?
>
No good
We hit a warning when shutting down the nbd connection because we have irq's
disabled. We don't really need to do the shutdown under the lock, just clear
the nbd->sock. So do the shutdown outside of the irq. This gets rid of the
warning.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
This moves NBD over to using blkmq, which allows us to get rid of the NBD
wide queue lock and the async submit kthread. We will start with 1 hw
queue for now, but I plan to add multiple tcp connection support in the
future and we'll fix how we set the hwqueue's.
Signed-off-by: Josef Bacik <
Apologies if you are getting this a second time, it appears vger ate my last
submission.
--
This is a patch series aimed at bringing NBD into 2016. The two big components
of this series is converting nbd over to using blkmq and
In preparation for some future changes, change a few of the state bools over to
normal bits to set/clear properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/block/n
Instead of rolling our own timer, just utilize the blk mq req timeout and do the
disconnect if any of our commands timeout.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 51 ++-
1 file changed, 14 insertions(+), 37 del
In preparation for some future changes, change a few of the state bools over to
normal bits to set/clear properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 18 ++
1 file changed, 10 insertions(+), 8 deletions(-)
diff --git a/drivers/block/n
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 267
1 file changed, 143 insertions(+), 124 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4c6dd1a..4aa45ed
We hit a warning when shutting down the nbd connection because we have irq's
disabled. We don't really need to do the shutdown under the lock, just clear
the nbd->sock. So do the shutdown outside of the irq. This gets rid of the
warning.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 267
1 file changed, 143 insertions(+), 124 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4c6dd1a..4aa45ed
This moves NBD over to using blkmq, which allows us to get rid of the NBD
wide queue lock and the async submit kthread. We will start with 1 hw
queue for now, but I plan to add multiple tcp connection support in the
future and we'll fix how we set the hwqueue's.
Signed-off-by: Josef Bacik <
On 09/09/2016 04:55 PM, Wouter Verhelst wrote:
> On Fri, Sep 09, 2016 at 04:36:07PM -0400, Josef Bacik wrote:
>> On 09/09/2016 04:02 PM, Wouter Verhelst wrote:
> [...]
>>> I see some practical problems with this:
>>> - You removed the pid attribute from sysfs (unless
On 09/15/2016 09:17 AM, Wouter Verhelst wrote:
> On Thu, Sep 15, 2016 at 01:44:29PM +0100, Alex Bligh wrote:
>>
>>> On 15 Sep 2016, at 13:41, Christoph Hellwig wrote:
>>>
>>> On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote:
That's probably right in the case of
On 09/09/2016 04:02 PM, Wouter Verhelst wrote:
> Hi Josef,
>
> On Thu, Sep 08, 2016 at 05:12:05PM -0400, Josef Bacik wrote:
>> Apologies if you are getting this a second time, it appears vger ate my las
On 09/29/2016 05:52 AM, Wouter Verhelst wrote:
> Hi Josef,
>
> On Wed, Sep 28, 2016 at 04:01:32PM -0400, Josef Bacik wrote:
>> NBD can become contended on its single connection. We have to serialize all
>> writes and we can only process one read response at a time. Fi
On 09/29/2016 12:41 PM, Wouter Verhelst wrote:
> On Thu, Sep 29, 2016 at 10:03:50AM -0400, Josef Bacik wrote:
>> So think of it like normal disks with multiple channels. We don't send
>> flushes
>> down all the hwq's to make sure they are clear, we l
On 10/07/2016 02:36 PM, Pavel Machek wrote:
> On Thu 2016-09-15 10:43:54, Josef Bacik wrote:
>> NBD can become contended on its single connection. We have to serialize all
>> writes and we can only process one read response at a time. Fix this by
>> allowing userspac
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
V2->V3:
-Fixed a problem with the tag used for the requests.
-Rebased onto the patch that enables async submit.
V1->V2:
-Dropped the index from nbd_cmd and just used the hctx->queue_num as HCH
suggested
-Added th
On 10/03/2016 07:34 AM, Alex Bligh wrote:
>
>> On 3 Oct 2016, at 08:57, Christoph Hellwig wrote:
>>
>>> Can you clarify what you mean by that? Why is it an "odd flush
>>> definition", and how would you "properly" define it?
>>
>> E.g. take the defintion from NVMe which also
-process cases.
Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
V4->V5:
-Updated the multi-connection flag to be an actual unused value.
V3->V4:
-Fixed a problem where fast completes (or early completes) would crash because
we were still accessing the bio's on the submit side.
-
If an app exits before running NBD_DO_IT but after adding sockets we can
end up not being allowed to do a new nbd device. Fix this by making
NBD_CLEAR_SOCK reset the setup_task.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 1 +
1 file changed, 1 insertion(+)
diff
m>
>>
>> Reviewed-by: Omar Sandoval <osan...@fb.com>
>>
>> Compile-reviewed only :) Josef can probably test it if he cares
>> enough,
>> but it looks right.
>
> Look
The old maintainers email is bouncing and I've rewritten most of this
driver in the recent months. Also add linux-block to the mailinglist
and remove the old tree, I will send patches through the linux-block
tree. Thanks,
Signed-off-by: Josef Bacik <jba...@fb.com>
---
MAINTAINERS | 4 +
If we have large devices (say like the 40t drive I was trying to test with) we
will end up overflowing the int arguments to nbd_set_size and not get the right
size for our device. Fix this by using loff_t everywhere so I don't have to
think about this again. Thanks,
Signed-off-by: Josef Bacik
These 4 patches are to fix up various regressions and problems in NBD. The
ERESTARTSYS is the biggest patch but has been pretty well tested with a debug
patch that forced the behavior to happen. Everything else is relatively small,
and the queue timeout patch is a regression from last cycle.
From: Josef Bacik <jba...@fb.com>
We can submit IO in a processes context, which means there can be
pending signals. This isn't a fatal error for NBD, but it does require
some finesse. If the signal happens before we transmit anything then we
are ok, just requeue the request and
From: Josef Bacik <jba...@fb.com>
We've been relying on the block layer to assume rq->errors being set
translates into -EIO. I noticed in testing that sometimes this isn't
true, and really there's not much of a reason to have a counter instead
of just using -EIO. So set it properly so
From: Josef Bacik <jba...@fb.com>
We can't just set the timeout on the tagset, we have to set it on the
queue as it would have been setup already at this point.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 5 -
1 file changed, 4 insertions(+), 1 delet
ped state by ext4_commit_super().
[mlin: ported to 4.11-rc2]
Signed-off-by: Ratna Manoj Bolla <manoj...@gmail.com
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/bl
Hey sorry I just got back from LSF, I’ll look at this in the morning. Thanks,
Josef
On 3/22/17, 4:48 PM, "Ming Lin" wrote:
On Mon, Mar 20, 2017 at 3:58 PM, Ming Lin wrote:
> From: Ratna Manoj Bolla
>
> When a filesystem is mounted on a
For ease of management it would be nice for users to specify that the
device node for a nbd device is destroyed once it is disconnected and
there are no more users. Add a client flag and enable this operation to
happen.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block
In order to support deleting the device on disconnect we need to
refcount the actual nbd_device struct. So add the refcounting framework
and change how we free the normal devices at rmmod time so we can catch
reference leaks.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block
This patchset adds a new netlink configuration interface to NBD as well as a
bunch of enhancments around path failures. The patches provide the following
enhancemnts to NBD
- Netlink configuration interface that doesn't leave a userspace application
waiting in kernel space for the device to
> On Apr 6, 2017, at 5:01 PM, Josef Bacik <jo...@toxicpanda.com> wrote:
>
> This patchset adds a new netlink configuration interface to NBD as well as a
> bunch of enhancments around path failures. The patches provide the following
> enhancemnts to NBD
>
> - Netl
for less than the dead connection timeout time and bring it
back up and everything resumes gracefully.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 63 +---
include/uapi/linux/nbd-netlink.h | 1 +
2 files changed, 60 inse
replace dead connections with new more lively
connections.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 141 +++
include/uapi/linux/nbd-netlink.h | 1 +
2 files changed, 142 insertions(+)
diff --git a/drivers/block/n
one we mark it
as dead and send all IO destined for that socket to one of the other healthy
sockets. Any outstanding requests that were on the dead socket will timeout and
be re-submitted properly.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
than are preallocated at insmod time. We also have gained the ability
to not specify a particular device and be provided one for us so that
userspace doesn't need to find a free device to configure.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
the requests. So instead make the ioctl only shutdown
the sockets and make it so that we only ever run nbd_clear_que from the
device teardown.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/block/n
to configure the device. This has
a nice side-effect of fixing a long standing problem where you could end
up with a half-configured nbd device that needed to be "disconnected" in
order to be usable again. Now once we close our device the
configuration will be discarded.
Signed-off-by: Josef
In preparation for the upcoming netlink interface we need to not rely on
already having the bdev for the NBD device we are doing operations on.
Instead of passing the bdev around, just use it in places where we know
we already have the bdev.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
d
Allow users to query the status of existing nbd devices. Right now this
only returns whether or not the device is connected, but could be
extended in the future to include more information.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c
Provide a mechanism to notify userspace that there's been a link problem
on a NBD device. This will allow userspace to re-establish a connection
and provide the new socket to the device without disrupting the device.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block
> On Apr 17, 2017, at 11:59 AM, Jens Axboe <ax...@fb.com> wrote:
>
> On 04/17/2017 09:57 AM, Jens Axboe wrote:
>> On 04/17/2017 09:55 AM, Jens Axboe wrote:
>>> On 04/17/2017 07:44 AM, Josef Bacik wrote:
>>>> NBD doesn't care about limiting the segmen
NBD doesn't care about limiting the segment size, let the user push the
largest bio's they want. This allows us to control the request size
solely through max_sectors_kb.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 2 ++
1 file changed, 2 insertions(+)
diff
I lack the basic understanding of what segments mean, so we were being
limited to 512kib requests even with higher max_sectors sizes set.
Setting the maximum number of segments to unlimited allows us to
actually have arbitrarily large IO's go through NBD.
Signed-off-by: Josef Bacik <jba...@fb.
On Wed, 2017-03-08 at 11:07 +0100, Wouter Verhelst wrote:
> On Tue, Feb 28, 2017 at 11:57:11AM -0500, Josef Bacik wrote:
> >
> > The existing ioctl interface for configuring NBD devices is a bit
> > cumbersome and hard to extend. The other problem is we leave a
> > us
NBD doesn't care about limiting the segment size, let the user push the
largest bio's they want. This allows us to control the request size
solely through max_sectors_kb.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 1 +
1 file changed, 1 insertion(+)
diff
Oh shit the default timeout is 0 if you don't set it in the client. Use the
timeout option with nbd client and it should fix it for you. I'll send
something up to make this a sane default. Thanks,
Josef
Sent from my iPhone
> On Jul 21, 2017, at 8:15 AM, Adam Borowski
ing@redhat.com>
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 76e5f8f..9a6d34e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -20
rivers/block/nbd.c | 1 +
> 1 file changed, 1 insertion(+)
>
Oops, thanks Ilya
Reviewed-by: Josef Bacik <jba...@fb.com>
Josef
--
Check out the vibrant tech community on one of the world's most
engaging tech
On Tue, May 23, 2017 at 04:27:56PM +0200, Ilya Dryomov wrote:
> On Tue, May 23, 2017 at 4:16 PM, Jens Axboe <ax...@fb.com> wrote:
> > On 05/23/2017 08:14 AM, Josef Bacik wrote:
> >> On Tue, May 23, 2017 at 12:38:31PM +0200, Ilya Dryomov wrote:
> >>> nbd_con
On Tue, May 23, 2017 at 05:49:54PM +0200, Ilya Dryomov wrote:
> There is nothing to clear -- nbd_device has just been allocated.
> Fold nbd_reset() into its other caller, nbd_config_put().
>
> Signed-off-by: Ilya Dryomov <idryo...@gmail.com>
Reviewed-by: Josef Bacik <j
On Tue, May 23, 2017 at 05:49:55PM +0200, Ilya Dryomov wrote:
> nbd_config is allocated in nbd_alloc_config(), but never freed.
>
> Fixes: 5ea8d10802ec ("nbd: separate out the config information")
> Signed-off-by: Ilya Dryomov <idryo...@gmail.com>
Reviewed-by: Josef B
> Signed-off-by: Shaun McDowell <shaunjmcdow...@gmail.com>
Looks good to me, you can add
Reviewed-by: Josef Bacik <jba...@fb.com>
Thanks,
Josef
--
Check out the vibrant tech community on one of the world
was setup using netlink this caused us to spew every
time fio called this ioctl. Since we don't have special handling, just
error out for any non-nbd specific ioctl's that come in. This fixes the
spew.
Signed-off-by: Josef Bacik <jba...@fb.com>
---
drivers/block/nbd.c | 6 ++
1 file c
82 matches
Mail list logo