Re: [PATCH 1/9] blk-mq: don't call into depth limiting for reserved tags

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:19AM -0600, Jens Axboe wrote: > It's not useful, they are internal and/or error handling recovery > commands. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandoval > Signed-off-by: Jens Axboe > --- > block/blk-mq.c | 6 -- >

Re: [PATCH 2/9] bfq-iosched: don't worry about reserved tags in limit_depth

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:20AM -0600, Jens Axboe wrote: > Reserved tags are used for error handling, we don't need to > care about them for regular IO. The core won't call us for these > anyway. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandoval &g

Re: [PATCH 3/9] bfq: calculate shallow depths at init time

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:21AM -0600, Jens Axboe wrote: > It doesn't change, so don't put it in the per-IO hot path. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandoval > Signed-off-by: Jens Axboe > --- &g

Re: [PATCH 4/9] bfq-iosched: remove unused variable

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:22AM -0600, Jens Axboe wrote: > bfqd->sb_shift was attempted used as a cache for the sbitmap queue > shift, but we don't need it, as it never changes. Kill it with fire. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandoval > Si

Re: [PATCH 5/9] sbitmap: fix missed wakeups caused by sbitmap_queue_get_shallow()

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:23AM -0600, Jens Axboe wrote: > From: Omar Sandoval > > The sbitmap queue wake batch is calculated such that once allocations > start blocking, all of the bits which are already allocated must be > enough to fulfill the batch counters of all of

Re: [PATCH 6/9] sbitmap: warn if using smaller shallow depth than was setup

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:24AM -0600, Jens Axboe wrote: > From: Omar Sandoval > > Make sure the user passed the right value to > sbitmap_queue_min_shallow_depth(). An unlucky bisect that lands between this change and the BFQ/Kyber changes is going to trigger this warning. We sh

Re: [PATCH 7/9] bfq-iosched: update shallow depth to smallest one used

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:25AM -0600, Jens Axboe wrote: > If our shallow depth is smaller than the wake batching of sbitmap, > we can introduce hangs. Ensure that sbitmap knows how low we'll go. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandoval > Signe

Re: [PATCH 8/9] kyber-iosched: update shallow depth when setting up hardware queue

2018-05-10 Thread Omar Sandoval
On Thu, May 10, 2018 at 10:24:26AM -0600, Jens Axboe wrote: > We don't expect the async depth to be smaller than the wake batch > count for sbitmap, but just in case, inform sbitmap of what shallow > depth kyber may use. > > Acked-by: Paolo Valente Reviewed-by: Omar Sandov

Re: [PATCH blktests] nvme: Add command timeout injection test

2018-05-14 Thread Omar Sandoval
On Mon, May 14, 2018 at 02:02:37PM -0600, Keith Busch wrote: > This test will run a background IO process and inject an admin command > with a very short timeout that is all but guaranteed to expire without > a completion: the async event request. Thanks, a few comments below. > Signed-off-by: Ke

Re: [PATCHv2 blktests] nvme: Add command timeout injection test

2018-05-14 Thread Omar Sandoval
On Mon, May 14, 2018 at 02:42:41PM -0600, Keith Busch wrote: > This test will run a background IO process and inject an admin command > with a very short timeout that is all but guaranteed to expire without > a completion: the async event request. > > Signed-off-by: Keith Busch > --- > v1 -> v2:

Re: [PATCH blktests] Documentation: document prerequisite scriptlets

2018-05-21 Thread Omar Sandoval
On Mon, May 14, 2018 at 01:40:42PM +0200, Johannes Thumshirn wrote: > The config file is bash and it gets sourced, so all bash magic is > doable in there as well. Document it so others don't have to > re-discover this gem as well. I'm supportive of this... > Signed-off-by: Johannes Thumshirn > -

Re: [PATCH 00/10] Misc block layer patches for bcachefs

2018-05-21 Thread Omar Sandoval
On Mon, May 21, 2018 at 03:11:08PM +, Bart Van Assche wrote: > On Sun, 2018-05-20 at 19:58 -0400, Kent Overstreet wrote: > > On Sun, May 20, 2018 at 11:40:45PM +, Bart Van Assche wrote: > > > On Sun, 2018-05-20 at 19:21 -0400, Kent Overstreet wrote: > > > > I really have better things to do

Re: [PATCH] block: kyber: make kyber more friendly with merging

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 10:48:29PM +0800, Jianchao Wang wrote: > Currently, kyber is very unfriendly with merging. kyber depends > on ctx rq_list to do merging, however, most of time, it will not > leave any requests in ctx rq_list. This is because even if tokens > of one domain is used up, kyber w

Re: [PATCH blktests] blktests: use consistent helper returns

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 03:36:11AM -0400, Chaitanya Kulkarni wrote: > This is a cleanup patch which uses consistent return values > for helper functions. > > Signed-off-by: Chaitanya Kulkarni Thanks, applied.

Re: [PATCH blktests] block/020: test IO req allocation

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 07:12:49PM +0800, Ming Lei wrote: > IO request allocation may hang for ever if the allocation process > migrages. This test covers the request allocation code path. > > The following patch can fix this issue on linus kernel: > > https://marc.info/?l=linux-block&m=152671586

Re: [PATCH blktests] Documentation: document prerequisite scriptlets

2018-05-22 Thread Omar Sandoval
On Tue, May 22, 2018 at 09:30:24AM +0200, Johannes Thumshirn wrote: > On Mon, May 21, 2018 at 11:29:04AM -0700, Omar Sandoval wrote: > > But I'm curious about this specific example. Is this not mounted for you > > automatically? I'm guessing systemd does it for me on my s

Re: [PATCH V2] blk-mq: avoid to starve tag allocation after allocation process migrates

2018-05-23 Thread Omar Sandoval
gt; This patch fixes one request allocation hang issue, which can be > triggered easily in case of very low nr_request. > > Cc: > Cc: Omar Sandoval > Signed-off-by: Ming Lei > --- > > V2: > fix build failure > > block/blk-mq-tag.c | 13 +++

Re: [PATCH V2] blk-mq: avoid to starve tag allocation after allocation process migrates

2018-05-23 Thread Omar Sandoval
On Wed, May 23, 2018 at 05:32:31PM +0800, Ming Lei wrote: > On Tue, May 22, 2018 at 09:59:17PM -0600, Jens Axboe wrote: > > On 5/19/18 1:44 AM, Ming Lei wrote: > > > When the allocation process is scheduled back and the mapped hw queue is > > > changed, do one extra wake up on orignal queue for com

Re: [PATCH V2] blk-mq: avoid to starve tag allocation after allocation process migrates

2018-05-23 Thread Omar Sandoval
On Thu, May 24, 2018 at 06:09:51AM +0800, Ming Lei wrote: > On Thu, May 24, 2018 at 1:48 AM, Omar Sandoval wrote: > > On Wed, May 23, 2018 at 05:32:31PM +0800, Ming Lei wrote: > >> On Tue, May 22, 2018 at 09:59:17PM -0600, Jens Axboe wrote: > >> > On 5/19/18 1:44 AM,

Re: [PATCH] nbd: set discard granularity properly

2018-05-23 Thread Omar Sandoval
On Wed, May 23, 2018 at 01:35:59PM -0400, Josef Bacik wrote: > From: Josef Bacik > > For some reason we had discard granularity set to 512 always even when > discards were disabled. Fix this by having the default be 0, and then > if we turn it on set the discard granularity to the blocksize. >

Re: [PATCH V3] blk-mq: avoid to starve tag allocation after allocation process migrates

2018-05-23 Thread Omar Sandoval
gt; > This patch fixes one request allocation hang issue, which can be > triggered easily in case of very low nr_request. Thanks, Ming, this looks better. One comment below. > Cc: > Cc: Omar Sandoval > Signed-off-by: Ming Lei > --- > V3: > - fix comments as suggeste

Re: [PATCH V4] blk-mq: avoid to starve tag allocation after allocation process migrates

2018-05-24 Thread Omar Sandoval
gt; > This patch fixes one request allocation hang issue, which can be > triggered easily in case of very low nr_request. > > Cc: > Cc: Omar Sandoval Reviewed-by: Omar Sandoval The longer explanation you gave would be nice to have in the commit message, but maybe Jens can add

Re: [PATCH blktests] Fix block/011 to not use sysfs for device disabling

2018-05-29 Thread Omar Sandoval
On Mon, May 21, 2018 at 08:08:21AM -0600, Keith Busch wrote: > On Mon, May 21, 2018 at 02:37:56AM -0400, Yi Zhang wrote: > > Hi Keith > > I tried this patch on my R730 Server, but it lead to system hang after > > setpci, could you help check it, thanks. > > > > Console log: > > storageqe-62 login

Re: [PATCH] block/020: not pass shared_tags to null_blk

2018-05-29 Thread Omar Sandoval
On Wed, May 30, 2018 at 01:10:38PM +0800, Ming Lei wrote: > The issue isn't related with shared tag, and it can be > triggered in case 'shared_tags' is set as 0. > > Remove this so that this test can be run in older kernel, and > avoid to cause misunderstanding. Thanks, Ming, applied. > Signed-o

Re: [blktests PATCHv2] Fix block/011 to not use sysfs for device disabling

2018-06-04 Thread Omar Sandoval
On Mon, Jun 04, 2018 at 04:51:41PM -0600, Keith Busch wrote: > The PCI sysfs interface may not be a dependable method for toggling the > PCI device state to trigger the timeouts. This patch goes directly to > the config space to make device failure occur. > > Signed-off-by: Keith Busch > --- > v1

Re: [PATCH] block/021: test setting 'nr_requests'

2018-06-11 Thread Omar Sandoval
On Sat, Jun 02, 2018 at 01:02:04PM +0800, Ming Lei wrote: > 'nr_requests' can be a bit different for 'none' scheduler, > and especially current blk-mq can only decrease this value > for 'none.' > > The patch of 'blk-mq: fix read/write 'nr_requests' in case of 'none' > scheduler' > has been sent o

Re: [PATCH blktests] check: add command line switch to test device drivers only

2018-06-25 Thread Omar Sandoval
On Tue, Jun 12, 2018 at 11:57:06AM +0200, Johannes Thumshirn wrote: > Sometimes it's useful to only run tests which exercise a device > special driver to verify a patch for the driver doesn't introduce a > regression. > > Running the whole test-suite is just a waste of time in this case, so > prov

Re: [PATCH 0/9] blktests: Re-enable shellcheck warnings

2018-06-25 Thread Omar Sandoval
On Tue, Jun 19, 2018 at 01:23:44PM -0700, Bart Van Assche wrote: > Hello Omar, > > Since I noticed that several useful shellcheck warnings are suppressed in the > blktests project, I came up with this patch series that reenables all > shellcheck warnings and also suppresses false positive shellche

Re: [PATCH blktests 13/15] src/Makefile: Fix pattern rule

2018-06-25 Thread Omar Sandoval
On Fri, Jun 22, 2018 at 03:19:44PM -0700, Bart Van Assche wrote: > According to the GNU make documentation, the syntax for pattern rules > is as follows: > > +: > That's because this is a static pattern rule: https://www.gnu.org/software/make/manual/html_node/Static-Usage.html So callin

Re: [PATCH blktests 00/15] Add SRP initiator driver tests

2018-06-26 Thread Omar Sandoval
On Tue, Jun 26, 2018 at 08:24:51AM -0700, Bart Van Assche wrote: > On 06/26/18 00:27, Johannes Thumshirn wrote: > > On Tue, Jun 26, 2018 at 02:16:26AM +, Bart Van Assche wrote: > > > On Tue, 2018-06-26 at 09:37 +0800, Ming Lei wrote: > > > > I run SRP test on Fedora 27, and not run any LIO spec

Re: [PATCH blktests 15/15] Add tests for the SRP initiator and target drivers

2018-06-26 Thread Omar Sandoval
On Tue, Jun 26, 2018 at 08:25:40AM -0700, Bart Van Assche wrote: > On 06/26/18 01:58, Johannes Thumshirn wrote: > > On Fri, Jun 22, 2018 at 03:19:46PM -0700, Bart Van Assche wrote: > > > tests/srp/functions | 1288 +++ > > > > Can you please move these into

Re: [PATCH blktests 00/15] Add SRP initiator driver tests

2018-06-26 Thread Omar Sandoval
On Fri, Jun 22, 2018 at 03:19:31PM -0700, Bart Van Assche wrote: > Hello Omar, > > As promised during LSF/MM, I have converted the srp-tests software to the > blktests framework. This patch series included all blktests patches I postd > a few days ago. Please consider this patch series for inclusi

Re: [PATCH 1/5] blk-mq: cleanup blk_mq_get_driver_tag()

2018-06-26 Thread Omar Sandoval
ay in 0c2a6fe4dc3e ("blk-mq: don't special case flush inserts for blk-mq-sched"). Reviewed-by: Omar Sandoval > Cc: Omar Sandoval > Cc: Andrew Jones > Cc: Bart Van Assche > Cc: Christoph Hellwig > Signed-off-by: Ming Lei > --- > block/blk-mq.c | 19 +

Re: [PATCH 2/5] blk-mq: don't pass **hctx to blk_mq_mark_tag_wait()

2018-06-26 Thread Omar Sandoval
On Mon, Jun 25, 2018 at 07:31:46PM +0800, Ming Lei wrote: > 'hctx' won't be changed at all, so not necessary to pass > '**hctx' to blk_mq_mark_tag_wait(). > > Cc: Andrew Jones > Cc: Christoph Hellwig > Cc: Omar Sandoval > Cc: Bart Van Assche Revi

Re: [PATCH] blk-mq: don't queue more if we get a busy return

2018-06-28 Thread Omar Sandoval
ler, if > we get a BUSY return. This should also be more optimal in terms of > letting requests stay in the scheduler for as long as possible, if we > get a BUSY due to the regular out-of-tags condition. Reviewed-by: Omar Sandoval > Signed-off-by: Jens Axboe > > diff --git

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-06-28 Thread Omar Sandoval
On Wed, Jun 27, 2018 at 02:49:08PM -0700, Bart Van Assche wrote: > This patch adds the following tests: > 001: Create and remove LUNs > 002: File I/O on top of multipath concurrently with logout and login (mq) > 003: File I/O on top of multipath concurrently with logout and login (sq) > 004: File I

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-03 Thread Omar Sandoval
On Fri, Jun 29, 2018 at 09:13:53AM -0700, Bart Van Assche wrote: > On 06/28/18 16:43, Omar Sandoval wrote: > > On Wed, Jun 27, 2018 at 02:49:08PM -0700, Bart Van Assche wrote: > > [ ... ] > > srp/002 (File I/O on top of multipath concurrently with logout and login > >

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-03 Thread Omar Sandoval
On Tue, Jul 03, 2018 at 12:49:13PM -0700, Omar Sandoval wrote: > On Fri, Jun 29, 2018 at 09:13:53AM -0700, Bart Van Assche wrote: > > On 06/28/18 16:43, Omar Sandoval wrote: > > > On Wed, Jun 27, 2018 at 02:49:08PM -0700, Bart Van Assche wrote: > > > [ ... ] > &

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-03 Thread Omar Sandoval
On Tue, Jul 03, 2018 at 12:50:10PM -0700, Omar Sandoval wrote: > On Tue, Jul 03, 2018 at 12:49:13PM -0700, Omar Sandoval wrote: > > On Fri, Jun 29, 2018 at 09:13:53AM -0700, Bart Van Assche wrote: > > > On 06/28/18 16:43, Omar Sandoval wrote: > > > > On Wed, Jun 27,

Re: [PATCH blktests v2] dir-locals.el: fix indenation with Emacs

2018-07-06 Thread Omar Sandoval
On Fri, Jul 06, 2018 at 10:21:28AM +0200, Johannes Thumshirn wrote: > A recent update of Emacs broke .dir-locals.el's shell indentation > setting. > > Also set 'sh-indendation' not just 'sh-basic-offset' for this. Another typo ;) fixed and applied, thanks. > Signed-off-by: Johannes Thumshirn >

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-06 Thread Omar Sandoval
On Wed, Jul 04, 2018 at 04:24:42PM +, Bart Van Assche wrote: > On Tue, 2018-07-03 at 14:39 -0700, Omar Sandoval wrote: > > Alright, I installed multipath-tools from source and the segfaults are > > gone, but I still don't get these symlinks. Instead, they show up as >

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-06 Thread Omar Sandoval
On Fri, Jul 06, 2018 at 02:21:44PM -0700, Omar Sandoval wrote: > On Wed, Jul 04, 2018 at 04:24:42PM +, Bart Van Assche wrote: > > On Tue, 2018-07-03 at 14:39 -0700, Omar Sandoval wrote: > > > Alright, I installed multipath-tools from source and the segfaults are > > &

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-06 Thread Omar Sandoval
On Fri, Jul 06, 2018 at 11:07:06PM +, Bart Van Assche wrote: > On Fri, 2018-07-06 at 16:03 -0700, Omar Sandoval wrote: > > Alright, I'm not getting the udev events at all (even after enabling > > CONFIG_DM_UEVENT in addition to everything mentioned in the README). I >

Re: [PATCH blktests v2 3/3] Add tests for the SRP initiator and target drivers

2018-07-06 Thread Omar Sandoval
This might also be useful: # /sbin/multipathd -d -s -v 3 start up read /etc/multipath.conf loading /lib64/multipath/libchecktur.so checker loading /lib64/multipath/libprioconst.so prioritizer foreign library "nvme" loaded successfully set open fds limit to 1048576/1048576 path chec

Re: [PATCH blktests v3 0/6] Add SRP initiator driver tests

2018-07-10 Thread Omar Sandoval
On Mon, Jul 09, 2018 at 03:56:37PM -0700, Bart Van Assche wrote: > Hello Omar, > > As promised during LSF/MM, I have converted the srp-tests software to the > blktests framework. Please consider this patch series for inclusion in the > official blktests repository. This patch series is also availa

Re: [PATCH 0/2] blktests: test ANA base support

2018-07-25 Thread Omar Sandoval
ivially renumber it here. > > From: linux-block-ow...@vger.kernel.org > on behalf of Hannes Reinecke > Sent: Tuesday, July 17, 2018 6:31 AM > To: Omar Sandoval > Cc: Christoph Hellwig; Sagi Grimberg; Keith Busch; > linux-n...@lists.infradead.org; linux-block@vger.kernel.org

Re: [PATCH 0/2] blktests: test ANA base support

2018-07-25 Thread Omar Sandoval
On Wed, Jul 25, 2018 at 07:27:35PM +, Chaitanya Kulkarni wrote: > > Thanks, Omar. Tests nvme/014 and nvme/015 had a pretty bad typo that I didn't notice last time: dd=/dev/urandom of="/dev/${nvmedev}n1" count=128000 bs=4k That should be dd if=/dev/urandom of="/dev/${nvmedev}n1" count=12800

Re: [PATCH] blktests: Add '--outdir' to store results in a different directory

2018-07-25 Thread Omar Sandoval
On Tue, Jul 17, 2018 at 03:27:50PM +0200, Hannes Reinecke wrote: > Adding an option '--outdir' to store results in a different > director so as not to clutter the git repository itself. > > Signed-off-by: Hannes Reinecke > --- > check | 14 ++ > 1 file changed, 10 insertions(+), 4 de

Re: [PATCH] block/023: performance test on queue creation & cleanup

2018-07-25 Thread Omar Sandoval
On Wed, Jul 04, 2018 at 01:29:56PM +0800, Ming Lei wrote: > SCSI may have lots of channels, targets or LUNs, so it may > take long time for creating and cleaning up queues. > > So introduce block/023 and uses null_blk to run this test > on both blk-mq and legacy mode, then compare both and check >

Re: [PATCH 0/2] blktests: test ANA base support

2018-07-26 Thread Omar Sandoval
On Thu, Jul 26, 2018 at 09:35:53AM -0700, James Smart wrote: > make sure this fix has been picked up in your kernel: > http://git.infradead.org/nvme.git/commit/6cdefc6e2ad52170f89a8d0e8b1a1339f91834dc > > deletes were broken otherwise and have the exact trace below. > > -- james Thanks, James, t

Re: [PATCH 0/2] blktests: test ANA base support

2018-07-26 Thread Omar Sandoval
On Thu, Jul 26, 2018 at 02:31:32PM -0700, Omar Sandoval wrote: > On Thu, Jul 26, 2018 at 09:35:53AM -0700, James Smart wrote: > > make sure this fix has been picked up in your kernel: > > http://git.infradead.org/nvme.git/commit/6cdefc6e2ad52170f89a8d0e8b1a1339f91834dc > > &g

[RFC PATCH 1/5] block: move call of scheduler's ->completed_request() hook

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval Commit 4bc6339a583c ("block: move blk_stat_add() to __blk_mq_end_request()") consolidated some calls using ktime_get() so we'd only need to call it once. Kyber's ->completed_request() hook also calls ktime_get(), so let's move it to the same plac

[RFC PATCH 0/5] kyber: better heuristics

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval Hello, I've spent the past few weeks experimenting with different heuristics for Kyber in order to deal with some edge cases we've hit here. This series is my progress so far, implementing less handwavy heuristics while keeping the same basic mechanisms. Patches 1

[RFC PATCH 2/5] block: export blk_stat_enable_accounting()

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval Kyber will need this in a future change if it is built as a module. Signed-off-by: Omar Sandoval --- block/blk-stat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-stat.c b/block/blk-stat.c index 175c143ac5b9..d98f3ad6794e 100644 --- a/block/blk-stat.c

[RFC PATCH 3/5] kyber: don't make domain token sbitmap larger than necessary

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval The domain token sbitmaps are currently initialized to the device queue depth or 256, whichever is larger, and immediately resized to the maximum depth for that domain (256, 128, or 64 for read, write, and other, respectively). The sbitmap is never resized larger than that

[RFC PATCH 5/5] kyber: add tracepoints

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval When debugging Kyber, it's really useful to know what latencies we've been having and how the domain depths have been adjusted. Add two tracepoints, kyber_latency and kyber_adjust, to record that. Signed-off-by: Omar Sandoval --- block/kyber-iosched.c

[RFC PATCH 4/5] kyber: implement improved heuristics

2018-08-09 Thread Omar Sandoval
From: Omar Sandoval Kyber's current heuristics have a few flaws: - It's based on the mean latency, but p99 latency tends to be more meaningful to anyone who cares about latency. The mean can also be skewed by rare outliers that the scheduler can't do anything about. -

Re: [RFC PATCH 1/5] block: move call of scheduler's ->completed_request() hook

2018-08-10 Thread Omar Sandoval
On Fri, Aug 10, 2018 at 10:59:37AM +0800, jianchao.wang wrote: > Hi Omar > > On 08/10/2018 04:26 AM, Omar Sandoval wrote: > > @@ -524,6 +524,9 @@ inline void __blk_mq_end_request(struct request *rq, > > blk_status_t error) > > blk_stat_add(rq, now); >

Re: [PATCH] block: remove duplicate initialization

2018-08-17 Thread Omar Sandoval
esent in struct request_queue. Reviewed-by: Omar Sandoval > Signed-off-by: Chaitanya Kulkarni > --- > block/blk-core.c | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/block/blk-core.c b/block/blk-core.c > index 21c7cb35d3b4..dee56c282efb 100644 > --- a/block/blk-core.

Re: [PATCH blktests 0/3] Add NVMeOF multipath tests

2018-08-23 Thread Omar Sandoval
On Thu, Aug 23, 2018 at 01:53:33AM +, Bart Van Assche wrote: > On Tue, 2018-08-21 at 08:46 +0200, Johannes Thumshirn wrote: > > On Mon, Aug 20, 2018 at 03:46:45PM +, Bart Van Assche wrote: > > > Moving these tests into the nvme directory is possible but will make it > > > harder to run the

Re: [PATCH] blktests: remove unused null_blk parameter for _init_null_blk in block/016

2018-08-28 Thread Omar Sandoval
On Sat, Aug 25, 2018 at 06:06:19PM +0800, Yi Zhang wrote: > Signed-off-by: Yi Zhang > --- > tests/block/016 | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/tests/block/016 b/tests/block/016 > index 0e6f2e1..e0a63ad 100755 > --- a/tests/block/016 > +++ b/tests/block/016 >

Re: [PATCH] null_blk: fix zoned support for non-rq based operation

2018-09-12 Thread Omar Sandoval
4b2a011948 ("null_blk: add zone support") I just added block/023 to blktests for this, so Tested-by: Omar Sandoval > Signed-off-by: Jens Axboe > > --- > > diff --git a/drivers/block/null_blk.h b/drivers/block/null_blk.h > index d81781f22dba..34e0030f0592

Re: [PATCH blktests 0/3] Add NVMeOF multipath tests

2018-09-12 Thread Omar Sandoval
On Thu, Aug 23, 2018 at 05:21:33PM -0700, Omar Sandoval wrote: > On Thu, Aug 23, 2018 at 01:53:33AM +, Bart Van Assche wrote: > > On Tue, 2018-08-21 at 08:46 +0200, Johannes Thumshirn wrote: > > > On Mon, Aug 20, 2018 at 03:46:45PM +, Bart Van Assche wrote: > > >

Re: [PATCH blktests 0/3] Add NVMeOF multipath tests

2018-09-18 Thread Omar Sandoval
On Tue, Sep 18, 2018 at 02:20:59PM -0700, Bart Van Assche wrote: > On 8/23/18 5:21 PM, Omar Sandoval wrote: > > On Thu, Aug 23, 2018 at 01:53:33AM +, Bart Van Assche wrote: > > > On Tue, 2018-08-21 at 08:46 +0200, Johannes Thumshirn wrote: > > > > On Mon, Aug 20,

Re: [PATCH blktests 0/3] Add NVMeOF multipath tests

2018-09-18 Thread Omar Sandoval
On Tue, Sep 18, 2018 at 05:02:47PM -0700, Bart Van Assche wrote: > On 9/18/18 4:24 PM, Omar Sandoval wrote: > > On Tue, Sep 18, 2018 at 02:20:59PM -0700, Bart Van Assche wrote: > > > Can you have a look at the updated master branch of > > > https://github.com/bvanassche

[PATCH] block: use nanosecond resolution for iostat

2018-09-21 Thread Omar Sandoval
From: Omar Sandoval Klaus Kusche reported that the I/O busy time in /proc/diskstats was not updating properly on 4.18. This is because we started using ktime to track elapsed time, and we convert nanoseconds to jiffies when we update the partition counter. However, this gets rounded down, so any

Re: [PATCH] block: use nanosecond resolution for iostat

2018-09-24 Thread Omar Sandoval
On Fri, Sep 21, 2018 at 08:27:17PM -0600, Jens Axboe wrote: > On 9/21/18 5:44 PM, Omar Sandoval wrote: > > From: Omar Sandoval > > > > Klaus Kusche reported that the I/O busy time in /proc/diskstats was not > > updating properly on 4.18. This is because we start

Re: [PATCH 1/4] block/loop: Don't grab "struct file" for vfs_getattr() operation.

2018-09-25 Thread Omar Sandoval
On Wed, Sep 26, 2018 at 12:26:46AM +0900, Tetsuo Handa wrote: > vfs_getattr() needs "struct path" rather than "struct file". > Let's use path_get()/path_put() rather than get_file()/fput(). Reviewed-by: Omar Sandoval > Signed-off-by: Tetsuo Handa > Reviewed

Re: [PATCH] blk-mq: I/O and timer unplugs are inverted in blktrace

2018-09-27 Thread Omar Sandoval
ed in blk-mq since 4.11. > > Cc: sta...@vger.kernel.org > Fixes: bd166ef183c2 ("blk-mq-sched: add framework for MQ capable IO > schedulers") Reviewed-by: Omar Sandoval > Signed-off-by: Ilya Dryomov > --- > block/blk-mq.c | 4 ++-- > 1 file changed, 2 inserti

[PATCH v2 0/5] kyber: better heuristics

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval Hi, This is my series to improve the heuristics used by Kyber. Patches 1 and 2 are preparation. Patch 3 is a minor optimization. Patch 4 is the main change, and includes a detailed description of the new heuristics. Patch 5 adds tracepoints for debugging. This is basically

[PATCH v2 1/5] block: move call of scheduler's ->completed_request() hook

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval Commit 4bc6339a583c ("block: move blk_stat_add() to __blk_mq_end_request()") consolidated some calls using ktime_get() so we'd only need to call it once. Kyber's ->completed_request() hook also calls ktime_get(), so let's move it to the same plac

[PATCH v2 2/5] block: export blk_stat_enable_accounting()

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval Kyber will need this in a future change if it is built as a module. Signed-off-by: Omar Sandoval --- block/blk-stat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/block/blk-stat.c b/block/blk-stat.c index 7587b1c3caaf..90561af85a62 100644 --- a/block/blk-stat.c

[PATCH v2 5/5] kyber: add tracepoints

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval When debugging Kyber, it's really useful to know what latencies we've been having, how the domain depths have been adjusted, and if we've actually been throttling. Add three tracepoints, kyber_latency, kyber_adjust, and kyber_throttled, to record that. Sign

[PATCH v2 4/5] kyber: implement improved heuristics

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval Kyber's current heuristics have a few flaws: - It's based on the mean latency, but p99 latency tends to be more meaningful to anyone who cares about latency. The mean can also be skewed by rare outliers that the scheduler can't do anything about. -

[PATCH v2 3/5] kyber: don't make domain token sbitmap larger than necessary

2018-09-27 Thread Omar Sandoval
From: Omar Sandoval The domain token sbitmaps are currently initialized to the device queue depth or 256, whichever is larger, and immediately resized to the maximum depth for that domain (256, 128, or 64 for read, write, and other, respectively). The sbitmap is never resized larger than that

[PATCH] kyber: fix integer overflow of latency targets on 32-bit

2018-09-28 Thread Omar Sandoval
From: Omar Sandoval NSEC_PER_SEC has type long, so 5 * NSEC_PER_SEC is calculated as a long. However, 5 seconds is 5,000,000,000 nanoseconds, which overflows a 32-bit long. Make sure all of the targets are calculated as 64-bit values. Fixes: 6e25cb01ea20 ("kyber: implement improved heuri

Re: [PATCH blktests 0/3] Add NVMeOF multipath tests

2018-10-05 Thread Omar Sandoval
On Thu, Sep 27, 2018 at 04:26:42PM -0700, Bart Van Assche wrote: > On Tue, 2018-09-18 at 17:18 -0700, Omar Sandoval wrote: > > On Tue, Sep 18, 2018 at 05:02:47PM -0700, Bart Van Assche wrote: > > > On 9/18/18 4:24 PM, Omar Sandoval wrote: > > > > On Tue, Sep 18, 2018

Re: [PATCH] blk-mq-debugfs: Also show requests that have not yet been started

2018-10-05 Thread Omar Sandoval
On Fri, Oct 05, 2018 at 08:18:00AM -0600, Jens Axboe wrote: > On 10/4/18 11:35 AM, Bart Van Assche wrote: > > When debugging e.g. the SCSI timeout handler it is important that > > requests that have not yet been started or that already have > > completed are also reported through debugfs. > > Than

[PATCH 02/13] swim: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval The only interesting thing here is that there may be two floppies (i.e., request queues) sharing the same controller, so we use the global struct swim_priv->lock to check whether the controller is busy. Compile-tested only. Cc: Finn Thain Cc: Laurent Vivier Signed-off

[PATCH 00/13] Convert floppy drivers to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Hi, This series converts the various floppy drivers to blk-mq. Save for the last one (floppy), they're compile-tested only. If I've Cc'd you, it's because I think you might be able to test the changes. Please test if you can, or let me know if there'

[PATCH 01/13] swim: fix cleanup on setup error

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval If we fail to allocate the request queue for a disk, we still need to free that disk, not just the previous ones. Additionally, we need to cleanup the previous request queues. Signed-off-by: Omar Sandoval --- drivers/block/swim.c | 13 +++-- 1 file changed, 11

[PATCH 04/13] swim3: end whole request on error

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval When swim3_end_request() gets passed an error, it seems that the intent is to end the whole request, but we're only ending the current segment. Signed-off-by: Omar Sandoval --- drivers/block/swim3.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --

[PATCH 03/13] swim3: add real error handling in setup

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval The driver doesn't have support for removing a device that has already been configured, but with more careful ordering we can avoid the need for that and make sure that we don't leak generic resources. Signed-off-by: Omar Sandoval --- drivers/block/sw

[PATCH 09/13] ataflop: fold headers into C file

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval atafd.h and atafdreg.h are only used from ataflop.c, so merge them in there. Signed-off-by: Omar Sandoval --- arch/m68k/include/asm/atafd.h| 13 - arch/m68k/include/asm/atafdreg.h | 80 -- drivers/block/ataflop.c | 83

[PATCH 12/13] floppy: end whole request on error

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval If floppy_end_request() gets passed an error, it should end the whole request, not just the current segment. Signed-off-by: Omar Sandoval --- drivers/block/floppy.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/block/floppy.c b/drivers

[PATCH 05/13] swim3: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Pretty simple conversion. To avoid extra churn, we keep the swim3_end_request() wrapper. grab_drive() could probably be replaced by some freeze/quiesce incantation, but I left it alone, and just used freeze/quiesce for eject. Compile-tested only. Cc: Benjamin Herrenschmidt

[PATCH 07/13] amiflop: clean up on errors during setup

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval The error handling in fd_probe_drives() doesn't clean up at all. Fix it up in preparation for converting to blk-mq. While we're here, get rid of the commented out amiga_floppy_remove(). Signed-off-by: Omar Sandoval --- drivers/block/amif

[PATCH 13/13] floppy: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval This driver likes to fetch requests from all over the place, so make queue_rq put requests on a list so that the logic stays the same. Tested with QEMU. Signed-off-by: Omar Sandoval --- drivers/block/floppy.c | 74 +- 1 file changed

[PATCH 10/13] ataflop: fix error handling during setup

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Move queue allocation next to disk allocation to fix a couple of issues: - If add_disk() hasn't been called, we should clear disk->queue before calling put_disk(). - If we fail to allocate a request queue, we still need to put all of the disks, not just the ones

[PATCH 08/13] amiflop: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Straightforward conversion, just use the existing amiflop_lock to serialize access to the controller. Compile-tested only. Cc: Laurent Vivier Signed-off-by: Omar Sandoval --- drivers/block/amiflop.c | 127 +++- 1 file changed, 48

[PATCH 11/13] ataflop: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval This driver is already pretty broken, in that it has two wait_events() (one in stdma_lock()) in request_fn. Get rid of the first one by freezing/quiescing the queue on format, and the second one by replacing it with stdma_try_lock(). The rest is straightforward. Compile

[PATCH 06/13] amiflop: fold headers into C file

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval amifd.h and amifdreg.h are only used from amiflop.c, and they're pretty small, so move the contents to amiflop.c and get rid of the .h files. This is preparation for adding a struct blk_mq_tag_set to struct amiga_floppy_struct. Signed-off-by: Omar Sandoval --- dr

Re: [PATCH 12/13] floppy: end whole request on error

2018-10-11 Thread Omar Sandoval
On Thu, Oct 11, 2018 at 12:39:51PM -0600, Jens Axboe wrote: > On 10/11/18 12:30 PM, Omar Sandoval wrote: > > From: Omar Sandoval > > > > If floppy_end_request() gets passed an error, it should end the whole > > request, not just the current segment. > > I don&#

[PATCH v2 00/11] Convert floppy drivers to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Hi, This series converts the various floppy drivers to blk-mq. Save for the last one (floppy), they're compile-tested only. If I've Cc'd you, it's because I think you might be able to test the changes. Please test if you can, or let me know if there'

[PATCH v2 01/11] swim: fix cleanup on setup error

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval If we fail to allocate the request queue for a disk, we still need to free that disk, not just the previous ones. Additionally, we need to cleanup the previous request queues. Signed-off-by: Omar Sandoval --- drivers/block/swim.c | 13 +++-- 1 file changed, 11

[PATCH v2 04/11] swim3: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval Pretty simple conversion. grab_drive() could probably be replaced by some freeze/quiesce incantation, but I left it alone, and just used freeze/quiesce for eject. Compile-tested only. Cc: Benjamin Herrenschmidt Signed-off-by: Omar Sandoval --- drivers/block/swim3.c | 163

[PATCH v2 02/11] swim: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval The only interesting thing here is that there may be two floppies (i.e., request queues) sharing the same controller, so we use the global struct swim_priv->lock to check whether the controller is busy. Compile-tested only. Cc: Finn Thain Cc: Laurent Vivier Signed-off

[PATCH v2 03/11] swim3: add real error handling in setup

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval The driver doesn't have support for removing a device that has already been configured, but with more careful ordering we can avoid the need for that and make sure that we don't leak generic resources. Signed-off-by: Omar Sandoval --- drivers/block/sw

[PATCH v2 05/11] amiflop: fold headers into C file

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval amifd.h and amifdreg.h are only used from amiflop.c, and they're pretty small, so move the contents to amiflop.c and get rid of the .h files. This is preparation for adding a struct blk_mq_tag_set to struct amiga_floppy_struct. Signed-off-by: Omar Sandoval --- dr

[PATCH v2 11/11] floppy: convert to blk-mq

2018-10-11 Thread Omar Sandoval
From: Omar Sandoval This driver likes to fetch requests from all over the place, so make queue_rq put requests on a list so that the logic stays the same. Tested with QEMU. Signed-off-by: Omar Sandoval --- drivers/block/floppy.c | 74 +- 1 file changed

  1   2   3   4   5   6   7   8   >