Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Martin K. Petersen
Hi Ming, > Please cook a patch for fixing the crash issue only, since we need > to backport the fix to stable kernel. I thought you were going to submit a V5 that addressed James' concerns? -- Martin K. Petersen Oracle Linux Engineering

Re: [PATCH v2 0/4] lockdep/crossrelease: Apply crossrelease to page locks

2017-12-05 Thread Michal Hocko
On Mon 04-12-17 23:45:17, Matthew Wilcox wrote: > On Tue, Dec 05, 2017 at 03:19:46PM +0900, Byungchul Park wrote: > > On 12/5/2017 2:46 PM, Byungchul Park wrote: > > > On 12/5/2017 2:30 PM, Matthew Wilcox wrote: > > > > On Mon, Dec 04, 2017 at 02:16:19PM +0900, Byungchul Park wrote: > > > > > For n

[PATCH] aio: Add memcg accounting of user used data

2017-12-05 Thread Kirill Tkhai
Currently, number of available aio requests may be limited only globally. There are two sysctl variables aio_max_nr and aio_nr, which implement the limitation and request accounting. They help to avoid the situation, when all the memory is eaten in-flight requests, which are written by slow block d

Re: [PATCH V15 00/22] mmc: Add Command Queue support

2017-12-05 Thread Linus Walleij
On Wed, Nov 29, 2017 at 4:47 PM, Ulf Hansson wrote: > I have applied patches 1->19 for next. Deferring patch 21->23 for a while. > > For those patches that was more or less the same as in v14, I added Linus' > ack. My ACK goes for the new set as well. > Hopefully we get some help for the comm

Re: [PATCH] SCSI: delay run queue if device is blocked in scsi_dev_queue_ready()

2017-12-05 Thread Holger Hoffstätte
On 12/05/17 06:16, Ming Lei wrote: > On Mon, Dec 04, 2017 at 11:48:07PM +, Holger Hoffstätte wrote: >> On Tue, 05 Dec 2017 06:45:08 +0800, Ming Lei wrote: >> >>> On Mon, Dec 04, 2017 at 03:09:20PM +, Bart Van Assche wrote: On Sun, 2017-12-03 at 00:31 +0800, Ming Lei wrote: > Fixes:

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Johannes Thumshirn
[ +Cc Omar ] Ming Lei writes: > Before commit 0df21c86bdbf ("scsi: implement .get_budget and .put_budget > for blk-mq"), we run queue after 3ms if queue is idle and SCSI device > queue isn't ready, which is done in handling BLK_STS_RESOURCE. After > commit 0df21c86bdbf is introduced, queue won't

Re: [PATCH] aio: Add memcg accounting of user used data

2017-12-05 Thread Michal Hocko
On Tue 05-12-17 13:00:54, Kirill Tkhai wrote: > Currently, number of available aio requests may be > limited only globally. There are two sysctl variables > aio_max_nr and aio_nr, which implement the limitation > and request accounting. They help to avoid > the situation, when all the memory is eat

Re: [PATCH 0/5] blkcg: Limit maximum number of aio requests available for cgroup

2017-12-05 Thread Oleg Nesterov
On 12/05, Kirill Tkhai wrote: > > Currently, aio_nr and aio_max_nr are global. Yeah, I too tried to complain 2 years ago... > In case of containers this > means that a single container may occupy all aio requests, which are > available in the system, and memory. let me quote my old emails... T

Re: [PATCH 0/5] blkcg: Limit maximum number of aio requests available for cgroup

2017-12-05 Thread Benjamin LaHaise
On Tue, Dec 05, 2017 at 04:19:56PM +0100, Oleg Nesterov wrote: > On 12/05, Kirill Tkhai wrote: > > > > Currently, aio_nr and aio_max_nr are global. > > Yeah, I too tried to complain 2 years ago... > > > In case of containers this > > means that a single container may occupy all aio requests, whic

Re: [PATCH] aio: Add memcg accounting of user used data

2017-12-05 Thread Kirill Tkhai
On 05.12.2017 18:15, Michal Hocko wrote: > On Tue 05-12-17 13:00:54, Kirill Tkhai wrote: >> Currently, number of available aio requests may be >> limited only globally. There are two sysctl variables >> aio_max_nr and aio_nr, which implement the limitation >> and request accounting. They help to av

Re: [PATCH 0/5] blkcg: Limit maximum number of aio requests available for cgroup

2017-12-05 Thread Jeff Moyer
Kirill Tkhai writes: >> I think you just need to account the completion ring. > > A request of struct aio_kiocb type consumes much more memory, than > struct io_event does. Shouldn't we account it too? Not in my opinion. The completion ring is the part that gets pinned for long periods of time.

Re: [PATCH] aio: Add memcg accounting of user used data

2017-12-05 Thread Michal Hocko
On Tue 05-12-17 18:34:59, Kirill Tkhai wrote: > On 05.12.2017 18:15, Michal Hocko wrote: > > On Tue 05-12-17 13:00:54, Kirill Tkhai wrote: > >> Currently, number of available aio requests may be > >> limited only globally. There are two sysctl variables > >> aio_max_nr and aio_nr, which implement t

Re: [PATCH 0/5] blkcg: Limit maximum number of aio requests available for cgroup

2017-12-05 Thread Tejun Heo
Hello, Jeff. On Tue, Dec 05, 2017 at 10:41:11AM -0500, Jeff Moyer wrote: > Kirill Tkhai writes: > > >> I think you just need to account the completion ring. > > > > A request of struct aio_kiocb type consumes much more memory, than > > struct io_event does. Shouldn't we account it too? > > Not

Re: [PATCH V15 00/22] mmc: Add Command Queue support

2017-12-05 Thread Ulf Hansson
On 5 December 2017 at 11:10, Linus Walleij wrote: > On Wed, Nov 29, 2017 at 4:47 PM, Ulf Hansson wrote: > >> I have applied patches 1->19 for next. Deferring patch 21->23 for a while. >> >> For those patches that was more or less the same as in v14, I added Linus' >> ack. > > My ACK goes for th

Re: [PATCH] aio: Add memcg accounting of user used data

2017-12-05 Thread Kirill Tkhai
On 05.12.2017 18:43, Michal Hocko wrote: > On Tue 05-12-17 18:34:59, Kirill Tkhai wrote: >> On 05.12.2017 18:15, Michal Hocko wrote: >>> On Tue 05-12-17 13:00:54, Kirill Tkhai wrote: Currently, number of available aio requests may be limited only globally. There are two sysctl variables >

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Bart Van Assche
On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > index db9556662e27..1816dd8259b3 100644 > --- a/drivers/scsi/scsi_lib.c > +++ b/drivers/scsi/scsi_lib.c > @@ -1967,6 +1967,8 @@ static bool scsi_mq_get_budget(struct blk_mq_hw_ctx

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Bart Van Assche
On Tue, 2017-12-05 at 15:29 +0100, Johannes Thumshirn wrote: > 1) Testing without the patch applied hangs the test forever as it >doesn't get killed after a specific timeout (I think this should be >solved in a common function). Hello Johannes, If a request queue got stuck then the proces

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Bart Van Assche
On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > No, do not mix two different things in one patch, especially the fix part > need to be backported to stable. > > The fix part should aim at V4.15, and the other part can be a V4.16 > stuff. Does this mean that you do not plan to post a v5 of yo

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:08:20PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > > index db9556662e27..1816dd8259b3 100644 > > --- a/drivers/scsi/scsi_lib.c > > +++ b/drivers/scsi/scsi_lib.c >

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:22:33PM +, Bart Van Assche wrote: > On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > > No, do not mix two different things in one patch, especially the fix part > > need to be backported to stable. > > > > The fix part should aim at V4.15, and the other part can

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Bart Van Assche
On Wed, 2017-12-06 at 00:28 +0800, Ming Lei wrote: > This is still a workaround for RESTART, see my comment before: > > https://marc.info/?l=linux-block&m=151217500929341&w=2 A quote from that e-mail: "The theory about using BLK_MQ_S_SCHED_RESTART in current way is that we mark it after req

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread James Bottomley
On Wed, 2017-12-06 at 00:38 +0800, Ming Lei wrote: > On Tue, Dec 05, 2017 at 04:22:33PM +, Bart Van Assche wrote: > > > > On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > > > > > > No, do not mix two different things in one patch, especially the > > > fix part need to be backported to sta

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 04:41:46PM +, Bart Van Assche wrote: > On Wed, 2017-12-06 at 00:28 +0800, Ming Lei wrote: > > This is still a workaround for RESTART, see my comment before: > > > > https://marc.info/?l=linux-block&m=151217500929341&w=2 > > A quote from that e-mail: "The theory abo

Re: [PATCH V4] scsi_debugfs: fix crash in scsi_show_rq()

2017-12-05 Thread Ming Lei
On Wed, Nov 15, 2017 at 08:04:49PM +0800, Ming Lei wrote: > On Wed, Nov 15, 2017 at 07:28:00PM +0900, James Bottomley wrote: > > On Wed, 2017-11-15 at 18:09 +0800, Ming Lei wrote: > > > On Tue, Nov 14, 2017 at 10:14:52AM -0800, James Bottomley wrote: > > > > > > > > On Tue, 2017-11-14 at 08:55 +08

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Bart Van Assche
On Wed, 2017-12-06 at 00:38 +0800, Ming Lei wrote: > On Tue, Dec 05, 2017 at 04:22:33PM +, Bart Van Assche wrote: > > On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > > > No, do not mix two different things in one patch, especially the fix part > > > need to be backported to stable. > > >

Re: [PATCH] kyber: fix another domain token wait queue hang

2017-12-05 Thread Omar Sandoval
On Mon, Dec 04, 2017 at 03:12:15PM -0800, Omar Sandoval wrote: > From: Omar Sandoval > > Commit 8cf466602028 ("kyber: fix hang on domain token wait queue") fixed > a hang caused by leaving wait entries on the domain token wait queue > after the __sbitmap_queue_get() retry succeeded, making that w

[PATCH v2 0/3] Show all commands in debugfs

2017-12-05 Thread Bart Van Assche
Hello Jens, While debugging an issue with the SCSI error handler I noticed that commands that got stuck in that error handler are not shown in debugfs. That is very annoying for anyone who relies on the information in debugfs for root-causing such an issue. Hence this patch series that makes sure

[PATCH v2 2/3] blk-mq-debugfs: Also show requests that have not yet been started

2017-12-05 Thread Bart Van Assche
When debugging e.g. the SCSI timeout handler it is important that requests that have not yet been started or that already have completed are also reported through debugfs. Fixes: commit 2720bab50258 ("blk-mq-debugfs: Show busy requests") Signed-off-by: Bart Van Assche Cc: Ming Lei Cc: Christoph

[PATCH v2 1/3] scsi: Fix a scsi_show_rq() NULL pointer dereference

2017-12-05 Thread Bart Van Assche
Avoid that scsi_show_rq() triggers a NULL pointer dereference if called after sd_uninit_command(). Swap the NULL pointer assignment and the mempool_free() call in sd_uninit_command() to make it less likely that scsi_show_rq() triggers a use-after-free. Note: even with these changes scsi_show_rq() c

[PATCH v2 3/3] scsi-mq-debugfs: Show more information

2017-12-05 Thread Bart Van Assche
Show the request result, request timeout and SCSI command flags. This information is very helpful when trying to figure out why a queue got stuck. An example of the information that is exported through debugfs: $ (cd /sys/kernel/debug/block && find -type f -print0 | xargs -0 grep ago) ./sda/hctx0/

Re: [PATCH 1/2] scsi-mq: Only show the CDB if available

2017-12-05 Thread Ming Lei
On Tue, Dec 05, 2017 at 08:43:49AM -0800, James Bottomley wrote: > On Wed, 2017-12-06 at 00:38 +0800, Ming Lei wrote: > > On Tue, Dec 05, 2017 at 04:22:33PM +, Bart Van Assche wrote: > > > > > > On Tue, 2017-12-05 at 13:00 +0800, Ming Lei wrote: > > > > > > > > No, do not mix two different th

Re: [PATCH] SCSI: run queue if SCSI device queue isn't ready and queue is idle

2017-12-05 Thread Ming Lei
On Wed, Dec 06, 2017 at 12:28:25AM +0800, Ming Lei wrote: > On Tue, Dec 05, 2017 at 04:08:20PM +, Bart Van Assche wrote: > > On Tue, 2017-12-05 at 15:52 +0800, Ming Lei wrote: > > > diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c > > > index db9556662e27..1816dd8259b3 100644 > >

[PATCH v2] kyber: fix another domain token wait queue hang

2017-12-05 Thread Omar Sandoval
From: Omar Sandoval Commit 8cf466602028 ("kyber: fix hang on domain token wait queue") fixed a hang caused by leaving wait entries on the domain token wait queue after the __sbitmap_queue_get() retry succeeded, making that wait entry a "dud" which won't in turn wake more entries up. However, we c