Re: [PATCH 5/6] blk-mq-debugfs: Show busy requests

2017-05-02 Thread Bart Van Assche
On Mon, 2017-05-01 at 17:17 -0700, Omar Sandoval wrote: > On Thu, Apr 27, 2017 at 08:54:36AM -0700, Bart Van Assche wrote: > > + if (q->queue_hw_ctx[rq->mq_ctx->index_hw] == ctx->hctx && > > This doesn't look right. ctx->index_hw is the index into hctx-

Re: [PATCH 6/6] blk-mq-debugfs: Add 'kick' operation

2017-05-02 Thread Bart Van Assche
On Mon, 2017-05-01 at 18:24 -0600, Jens Axboe wrote: > On 05/01/2017 06:19 PM, Omar Sandoval wrote: > > On Thu, Apr 27, 2017 at 08:54:37AM -0700, Bart Van Assche wrote: > > > Running a queue causes the block layer to examine the per-CPU and > > > hw queues but not

Re: [PATCH 4/6] mtip32xx: convert internal command issue to block IO path

2017-05-02 Thread Bart Van Assche
On Tue, 2017-05-02 at 09:25 -0600, Jens Axboe wrote: > On 05/02/2017 09:16 AM, Christoph Hellwig wrote: > > Any reason for the move from ->end_io_data to ->special? I thought > > that ->special was something we'd get rid of sooner or later now > > that we can have additional per-cmd data even for

Re: [PATCH 0/7] blk-mq: fix queue quiescing

2017-05-25 Thread Bart Van Assche
On Thu, 2017-05-25 at 17:09 +0800, Ming Lei wrote: > Another big issue is that 'srcu_struct' is very big, which shouldn't > be embedded into hctx, since we only have one real user of > BLK_MQ_F_BLOCKING. > > So I will fix that too. Hello Ming, Is something like the (untested) patch below

[PATCH 00/19] Block layer patches for kernel v4.13

2017-05-25 Thread Bart Van Assche
not yet had the time to post these. Please consider these patches for kernel v4.13. Thanks, Bart. Bart Van Assche (19): block: Avoid that blk_exit_rl() triggers a use-after-free block: Introduce queue flag QUEUE_FLAG_SCSI_PDU bsg: Check queue type before attaching to a queue pktcdvd: Check

[PATCH 02/19] block: Introduce queue flag QUEUE_FLAG_SCSI_PDU

2017-05-25 Thread Bart Van Assche
>From the context where a SCSI command is submitted it is not always possible to figure out whether or not the queue the command is submitted to has struct scsi_request as the first member of its private data. Hence introduce the flag QUEUE_FLAG_SCSI_PDU. Signed-off-by: Bart Van Ass

[PATCH 05/19] cdrom: Check private request size before attaching to a queue

2017-05-25 Thread Bart Van Assche
t;) Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Omar Sandoval <osan...@fb.com> Cc: linux-block@vger.kernel.org --- drivers/cdrom/cdrom.c | 4 1 file changed, 4 inserti

[PATCH 07/19] scsi: Make scsi_ioctl_reset() pass the request queue pointer to blk_rq_init()

2017-05-25 Thread Bart Van Assche
. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: James Bottomley <james.bottom...@hansenpartnership.com> ---

[PATCH 15/19] block: Check locking assumptions at runtime

2017-05-25 Thread Bart Van Assche
Instead of documenting the locking assumptions of most block layer functions as a comment, use lockdep_assert_held() to verify locking assumptions at runtime. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h.

[PATCH 06/19] nfsd: Check private request size before submitting a SCSI request

2017-05-25 Thread Bart Van Assche
truct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: J. Bruce Fields <bfie...@fieldses.org> Cc: Jeff Layton <jlay...@poochiereds.net> Cc: Jens Axboe <ax...@fb.com> Cc: Christoph Hellwig <h..

[PATCH 19/19] block: Make request operation type argument declarations consistent

2017-05-25 Thread Bart Van Assche
Instead of declaring the second argument of blk_*_get_request() as int and passing it to functions that expect an unsigned int, declare that second argument as unsigned int. Also because of consistency, rename that second argument from 'rw' into 'op'. Signed-off-by: Bart Van Assche <bart.van

[PATCH 13/19] blk-mq: Fix the comment above blk_mq_quiesce_queue()

2017-05-25 Thread Bart Van Assche
that comment. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming Lei <ming@redhat.com> --- block/blk-mq.c | 5 ++--- 1 file changed, 2 inser

[PATCH 08/19] block: Introduce request_queue.initialize_rq_fn()

2017-05-25 Thread Bart Van Assche
callback function to struct request_queue. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Jens Axboe <ax...@fb.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Omar Sandoval <osan...@fb.com> Cc: linux-block@vger.ke

[PATCH 03/19] bsg: Check queue type before attaching to a queue

2017-05-25 Thread Bart Van Assche
x BSG ioctl memory corruption") References: commit 82ed4db499b8 ("block: split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Om

[PATCH 11/19] blk-mq: Reduce blk_mq_hw_ctx size

2017-05-25 Thread Bart Van Assche
Since the srcu structure is rather large (184 bytes on an x86-64 system), only allocate it if needed. Reported-by: Ming Lei <ming@redhat.com> Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.c

[PATCH 10/19] blk-mq: Change blk_mq_hw_ctx.queue_rq_srcu into an array

2017-05-25 Thread Bart Van Assche
This patch does not change any functionality but makes the next patch easier to read. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming

[PATCH 01/19] block: Avoid that blk_exit_rl() triggers a use-after-free

2017-05-25 Thread Bart Van Assche
;scsi: allocate scsi_cmnd structures as part of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Acked-by: Tejun Heo <t...@kernel.org> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Jens Axboe <ax...@fb.com> Cc: Christoph Hellwig <h...@l

[PATCH 04/19] pktcdvd: Check queue type before attaching to a queue

2017-05-25 Thread Bart Van Assche
split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Jens Axboe <ax...@fb.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Omar Sandoval <osan...@fb.com> Cc: linux-

[PATCH 09/19] block: Make scsi_req_init() calls implicit

2017-05-25 Thread Bart Van Assche
it as a separate function. References: commit 82ed4db499b8 ("block: split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Cc: Jens Axboe <ax...@fb.com> Cc: Christoph Hellwig <h...@

[PATCH 16/19] block: Document what queue type each function is intended for

2017-05-25 Thread Bart Van Assche
Some functions in block/blk-core.c must only be used on blk-sq queues while others are safe to use against any queue type. Document which functions are intended for blk-sq queues and issue a warning if the blk-sq API is misused. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com&

[PATCH 12/19] blk-mq: Initialize a request before assigning a tag

2017-05-25 Thread Bart Van Assche
of in __blk_mq_finish_request(). This patch does not change the behavior of the block layer. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming

[PATCH 18/19] block: Constify disk_type

2017-05-25 Thread Bart Van Assche
The variable 'disk_type' is never modified so constify it. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming Lei <ming@redhat.com> ---

[PATCH 17/19] blk-mq: Document locking assumptions

2017-05-25 Thread Bart Van Assche
Document the locking assumptions in functions that modify blk_mq_ctx.rq_list to make it easier for humans to verify this code. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval &

[PATCH 14/19] block: Add a comment above queue_lockdep_assert_held()

2017-05-25 Thread Bart Van Assche
Add a comment above the queue_lockdep_assert_held() macro that explains the purpose of the q->queue_lock test. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb

Re: [PATCH 08/19] block: Introduce request_queue.initialize_rq_fn()

2017-05-30 Thread Bart Van Assche
On Sun, 2017-05-28 at 10:37 +0200, Christoph Hellwig wrote: > Oh, and btw - for the mq case we don't want to use the function > pointer directly in the queue, but in blk_mq_ops, so that we have > all the mq methods in one place. Hello Christoph, Are you sure of this? All other function pointers

Re: WARNING triggers at blk_mq_update_nr_hw_queues during nvme_reset_work

2017-05-30 Thread Bart Van Assche
, tag_set_list) > blk_mq_unfreeze_queue(q); > } > + > +void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues) > +{ > + mutex_lock(>tag_list_lock); > + __blk_mq_update_nr_hw_queues(set, nr_hw_queues); > + mutex_unlock(>tag_list_lock); > +} > EXPORT_SYMBOL_GPL(blk_mq_update_nr_hw_queues); These changes look fine to me, hence: Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH 00/31] SCSI patches for kernel v4.13.

2017-06-01 Thread Bart Van Assche
On Wed, 2017-05-24 at 22:04 -0400, Martin K. Petersen wrote: > > This patch series consists of the bug fixes I came up with during the > > past two months. Please consider these patches for kernel v4.13. > > No major objections from me. Although you may have to slice and dice the > series

Re: [PATCH v2 12/12] block: Rename blk_mq_rq_{to,from}_pdu()

2017-06-01 Thread Bart Van Assche
On Thu, 2017-06-01 at 08:08 +0200, Christoph Hellwig wrote: > On Wed, May 31, 2017 at 03:52:46PM -0700, Bart Van Assche wrote: > > Commit 6d247d7f71d1 ("block: allow specifying size for extra command > > data") added support for .cmd_size to blk-sq. Due to that patch t

Re: [PATCH v2 4/8] blk-mq: fix blk_mq_quiesce_queue

2017-05-30 Thread Bart Van Assche
On Tue, 2017-05-30 at 08:22 +0800, Ming Lei wrote: > On Sun, May 28, 2017 at 04:10:09PM +0000, Bart Van Assche wrote: > > I really would like to see the blk_queue_quiesced() tests as close as > > possible to > > the blk_mq_hctx_stopped() tests. But I agree that we n

Re: [PATCH v2 4/8] blk-mq: fix blk_mq_quiesce_queue

2017-05-30 Thread Bart Van Assche
On Sun, 2017-05-28 at 18:44 +0800, Ming Lei wrote: > First it is really a fix, and then a improvement, so could you tell me > where is wrong with the title and the description? Hello Ming, Can you explain me why you want to keep the blk_mq_stop_hw_queues() call in nvme_suspend_queue()? Since

[PATCH v2 4/4] blk-mq-debugfs: Add 'kick' operation

2017-05-31 Thread Bart Van Assche
Running a queue causes the block layer to examine the per-CPU and hw queues but not the requeue list. Hence add a 'kick' operation that also examines the requeue list. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes

[PATCH v2 3/4] blk-mq-debugfs: Show busy requests

2017-05-31 Thread Bart Van Assche
Requests that got stuck in a block driver are neither on blk_mq_ctx.rq_list nor on any hw dispatch queue. Make these visible in debugfs through the "busy" attribute. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes R

[PATCH v2 2/4] blk-mq-debugfs: Show requeue list

2017-05-31 Thread Bart Van Assche
When verifying whether or not a blk-mq driver forgot to kick the requeue list after having requeued a request it is important to be able to verify the contents of the requeue list. Hence export that list through debugfs. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Re

Re: [PATCH v2 3/4] blk-mq-debugfs: Show busy requests

2017-05-31 Thread Bart Van Assche
On Wed, 2017-05-31 at 14:43 -0700, Eduardo Valentin wrote: > On Wed, May 31, 2017 at 02:30:49PM -0700, Bart Van Assche wrote: > > +static void hctx_show_busy(struct request *rq, void *data, bool reserved) > > +{ > > + const struct show_busy_

[PATCH v2 2/6] block: Introduce queue flag QUEUE_FLAG_SCSI_PASSTHROUGH

2017-05-31 Thread Bart Van Assche
>From the context where a SCSI command is submitted it is not always possible to figure out whether or not the queue the command is submitted to has struct scsi_request as the first member of its private data. Hence introduce the flag QUEUE_FLAG_SCSI_PASSTHROUGH. Signed-off-by: Bart Van Ass

[PATCH v2 09/12] blk-mq: Document locking assumptions

2017-05-31 Thread Bart Van Assche
Document the locking assumptions in functions that modify blk_mq_ctx.rq_list to make it easier for humans to verify this code. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval &

[PATCH v2 00/12] More patches for kernel v4.13

2017-05-31 Thread Bart Van Assche
al_tag are no longer initialized twice. * Removed WARN_ON_ONCE(q->mq_ops) from blk_queue_bypass_end() because this function is used by both blk-sq and blk-mq. * Added several new patches, e.g. "block: Rename blk_mq_rq_{to,from}_pdu()". Please consider these patches for kernel v4.13.

[PATCH v2 10/12] block: Constify disk_type

2017-05-31 Thread Bart Van Assche
The variable 'disk_type' is never modified so constify it. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming Lei <ming@redhat.

[PATCH v2 03/12] block: Make most scsi_req_init() calls implicit

2017-05-31 Thread Bart Van Assche
it as a separate function. Keep the scsi_req_init() call in ide_prep_sense() because it follows a blk_rq_init() call. References: commit 82ed4db499b8 ("block: split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Hannes Reinecke &l

[PATCH v2 02/12] block: Introduce request_queue.initialize_rq_fn()

2017-05-31 Thread Bart Van Assche
callback function to struct request_queue. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> --- block/blk-core.c | 11 +-- include/linux/blkdev.h |

[PATCH v2 05/12] blk-mq: Initialize a request before assigning a tag

2017-05-31 Thread Bart Van Assche
() instead of relying on __blk_mq_finish_request(). Moving the initialization of .rq_flags is fine because all changes and tests of .rq_flags occur between blk_get_request() and finishing a request. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@l

[PATCH v2 0/6] Split scsi passthrough fields out of struct request sequel

2017-05-31 Thread Bart Van Assche
e_scsi_pdu() into blk_queue_scsi_passthrough(). - In the cdrom driver, moved the SCSI passthrough support test from register_cdrom() to cdrom_read_cdda_bpc(). Please consider these patches for kernel v4.13. Thanks, Bart. Bart Van Assche (6): block: Avoid that blk_exit_rl() triggers a use-after-free block: Intro

[PATCH v2 5/6] cdrom: Check SCSI passthrough support before reading audio

2017-05-31 Thread Bart Van Assche
the pcd nor the gdrom driver sets that pointer. References: commit 82ed4db499b8 ("block: split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar

[PATCH v2 1/6] block: Avoid that blk_exit_rl() triggers a use-after-free

2017-05-31 Thread Bart Van Assche
;scsi: allocate scsi_cmnd structures as part of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Acked-by: Tejun Heo <t...@kernel.org> Reviewed-by: Hannes Reinecke <h...@suse.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Cc: Jan Kara <j...

[PATCH v2 4/6] pktcdvd: Check queue type before attaching to a queue

2017-05-31 Thread Bart Van Assche
split scsi_request out of struct request") Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Hannes Reinecke <h...@suse.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Cc: Omar Sandoval <osan...@fb.com> --- drivers/block/pktcdvd.c | 5 + 1 f

Re: [PATCH v2 3/4] blk-mq-debugfs: Show busy requests

2017-05-31 Thread Bart Van Assche
On Wed, 2017-05-31 at 14:49 -0700, Eduardo Valentin wrote: > On Wed, May 31, 2017 at 09:45:54PM +0000, Bart Van Assche wrote: > > On Wed, 2017-05-31 at 14:43 -0700, Eduardo Valentin wrote: > > > On Wed, May 31, 2017 at 02:30:49PM -0700, Bart Van Assche wrote: > > > &g

[PATCH v2 01/12] block: Make request operation type argument declarations consistent

2017-05-31 Thread Bart Van Assche
. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <osan...@fb.com> Cc: Ming Lei <ming@redhat.com> --- block/blk-core.c | 13 +++-

[PATCH v2 12/12] block: Rename blk_mq_rq_{to,from}_pdu()

2017-05-31 Thread Bart Van Assche
ons. This patch does not change any functionality. Most of this patch has been generated by running the following shell command: sed -i 's/blk_mq_rq_to_pdu/blk_rq_to_pdu/g; s/blk_mq_rq_from_pdu/blk_rq_from_pdu/g' \ $(git grep -lE 'blk_mq_rq_(to|from)_pdu') Signed-off-by: Bart Van

[PATCH v2 08/12] block: Document what queue type each function is intended for

2017-05-31 Thread Bart Van Assche
Some functions in block/blk-core.c must only be used on blk-sq queues while others are safe to use against any queue type. Document which functions are intended for blk-sq queues and issue a warning if the blk-sq API is misused. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com&

[PATCH v2 04/12] block: Change argument type of scsi_req_init()

2017-05-31 Thread Bart Van Assche
Since scsi_req_init() works on a struct scsi_request, change the argument type into struct scsi_request *. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Hannes Reinecke <h...@suse.com> Reviewed-by:

[PATCH v2 06/12] block: Add a comment above queue_lockdep_assert_held()

2017-05-31 Thread Bart Van Assche
Add a comment above the queue_lockdep_assert_held() macro that explains the purpose of the q->queue_lock test. Signed-off-by: Bart Van Assche <bart.vanass...@sandisk.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Omar Sandoval <

Re: [PATCH v4 01/11] blk-mq: fix direct issue

2017-06-05 Thread Bart Van Assche
On Mon, 2017-06-05 at 23:59 +0800, Ming Lei wrote: > +static void __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, > + struct request *rq, > + blk_qc_t *cookie, bool may_sleep) > { > struct request_queue *q =

Re: [PATCH v4 07/11] blk-mq: use QUEUE_FLAG_QUIESCED to quiesce queue

2017-06-05 Thread Bart Van Assche
On Mon, 2017-06-05 at 23:59 +0800, Ming Lei wrote: > It is required that no dispatch can happen any more once > blk_mq_quiesce_queue() returns, and we don't have such requirement > on APIs of stopping queue. > > But blk_mq_quiesce_queue() still may not block/drain dispatch in the > the case of

Re: [PATCH v4 05/11] blk-mq: use the introduced blk_mq_unquiesce_queue()

2017-06-05 Thread Bart Van Assche
On Mon, 2017-06-05 at 23:59 +0800, Ming Lei wrote: > blk_mq_unquiesce_queue() is used for unquiescing the > queue explicitly, so replace blk_mq_start_stopped_hw_queues() > with it. Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v4 04/11] blk-mq: introduce blk_mq_unquiesce_queue

2017-06-05 Thread Bart Van Assche
introduce blk_mq_unquiesce_queue() and make it > as counterpart of blk_mq_quiesce_queue() explicitely. ^^^ explicitly? Anyway: Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v4 11/11] Revert "blk-mq: don't use sync workqueue flushing from drivers"

2017-06-05 Thread Bart Van Assche
> Also changes to cancel_delayed_work() in blk_mq_stop_hw_queue(). Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v4 10/11] blk-mq: clarify dispatch may not be drained/blocked by stopping queue

2017-06-05 Thread Bart Van Assche
On Mon, 2017-06-05 at 23:59 +0800, Ming Lei wrote: > +/* > + * We do not guarantee that dispatch can be drained or blocked > + * after blk_mq_stop_hw_queue() returns. Please use > + * blk_mq_quiesce_queue() for that requirement. > + */ Hello Ming, This is comment explains what

Re: [PATCH v5 08/12] blk-mq: use QUEUE_FLAG_QUIESCED to quiesce queue

2017-06-06 Thread Bart Van Assche
e race, because any queue restart in > other paths may break blk_mq_quiesce_queue(). With the introduced > flag of QUEUE_FLAG_QUIESCED, we don't need to depend on stopping queue > for quiescing any more. Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v5 11/12] blk-mq: clarify dispatch may not be drained/blocked by stopping queue

2017-06-06 Thread Bart Van Assche
On Tue, 2017-06-06 at 23:22 +0800, Ming Lei wrote: > BLK_MQ_S_STOPPED may not be observed in other concurrent I/O paths, > we can't guarantee that dispatching won't happen after returning > from the APIs of stopping queue. > > So clarify the fact and avoid potential misuse. Review

Re: [PATCH] elevator: fix truncation of icq_cache_name

2017-06-06 Thread Bart Van Assche
levator name, but only ELV_NAME_MAX + 5 characters were reserved > for it --- so in the case of a maximum-length elevator name, the 'q' > character in "_io_cq" would be truncated by snprintf(). Fix it by > reserving ELV_NAME_MAX + 6 characters instead. Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v5 03/12] blk-mq: move blk_mq_quiesce_queue() into include/linux/blk-mq.h

2017-06-06 Thread Bart Van Assche
On Tue, 2017-06-06 at 23:22 +0800, Ming Lei wrote: > We usually put blk_mq_*() into include/linux/blk-mq.h, so > move this API into there. Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v5 01/12] blk-mq: pass correct hctx to blk_mq_try_issue_directly

2017-06-06 Thread Bart Van Assche
ed Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH v5 02/12] blk-mq: fix direct issue

2017-06-06 Thread Bart Van Assche
s [2], the patch itself looks fine to me. Hence: Reviewed-by: Bart Van Assche <bart.vanass...@sandisk.com>

Re: [PATCH V3 10/12] scsi: sd_zbc: Limit zone write locking to sequential zones

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 19:06 +0900, Damien Le Moal wrote: > + * There is no write constraints on conventional zones. So any write ^^^ Should this have been "There are no"? > - if (sdkp->zones_wlock && > - test_and_set_bit(zno, sdkp->zones_wlock)) > + if

Re: [PATCH V3 06/12] scsi: sd_zbc: Rearrange code

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 19:06 +0900, Damien Le Moal wrote: > Rearrange sd_zbc_setup() to include use_16_for_rw and use_10_for_rw > assignments and move the calculation of sdkp->zone_shift together > with the assignment of the verified zone_blocks value in > sd_zbc_check_zone_size(). Both functions

Re: [PATCH V3 03/12] block: Add zoned block device information to request queue

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 19:06 +0900, Damien Le Moal wrote: > @@ -492,6 +497,10 @@ struct request_queue { > struct blk_integrity integrity; > #endif /* CONFIG_BLK_DEV_INTEGRITY */ > > +#ifdef CONFIG_BLK_DEV_ZONED > + struct blk_zonedzoned; > +#endif > + > #ifdef CONFIG_PM

Re: [PATCH V3 01/12] block: Fix declaration of blk-mq debugfs functions

2017-09-15 Thread Bart Van Assche
; structure to allow scheduler modules outside of the block directory to > define debugfs attributes. The title of this patch is "Fix declaration of ...". Should the title perhaps have been "Move declarations of ..."? Anyway: Reviewed-by: Bart Van Assche <bart.vanass...@wdc.com>

Re: [PATCH V3 02/12] block: Fix declaration of blk-mq scheduler functions

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 19:06 +0900, Damien Le Moal wrote: > The functions blk_mq_sched_free_hctx_data(), blk_mq_sched_try_merge(), > blk_mq_sched_try_insert_merge() and blk_mq_sched_request_inserted() are > all exported symbols but are declared only internally in > block/blk-mq-sched.h. Move these

Re: [PATCH V3 07/12] scsi: sd_zbc: Use well defined macros

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 19:06 +0900, Damien Le Moal wrote: > instead of open coding, use the min() macro to calculate a report zones > reply buffer length in sd_zbc_check_zone_size() and the round_up() > macro for calculating the number of zones in sd_zbc_setup(). Reviewed-by: Bart V

Re: [PATCH V4 08/10] block: allow to allocate req with RQF_PREEMPT when queue is preempt frozen

2017-09-14 Thread Bart Van Assche
On Thu, 2017-09-14 at 09:15 +0800, Ming Lei wrote: > On Wed, Sep 13, 2017 at 07:07:53PM +0000, Bart Van Assche wrote: > > On Thu, 2017-09-14 at 01:48 +0800, Ming Lei wrote: > > > No, that patch only changes blk_insert_cloned_request() which is used > > > by dm-rq

Re: [PATCH V4 08/10] block: allow to allocate req with RQF_PREEMPT when queue is preempt frozen

2017-09-13 Thread Bart Van Assche
On Thu, 2017-09-14 at 01:48 +0800, Ming Lei wrote: > No, that patch only changes blk_insert_cloned_request() which is used > by dm-rq(mpath) only, nothing to do with the reported issue during > suspend and sending SCSI Domain validation. There may be other ways to fix the SCSI domain validation

Re: [PATCH V4 08/10] block: allow to allocate req with RQF_PREEMPT when queue is preempt frozen

2017-09-13 Thread Bart Van Assche
On Thu, 2017-09-14 at 00:48 +0800, Ming Lei wrote: > Could you please let me know if your concern about race between > preempt freeze and blk_cleanup_queue() is addressed in my last > reply? Shouldn't we wait until v4.13-rc1 has been released before spending more energy on this? Certain patches

Re: [PATCH 1/5] block: don't call blk_mq_delay_run_hw_queue() in case of BLK_STS_RESOURCE

2017-09-15 Thread Bart Van Assche
On Sat, 2017-09-16 at 00:44 +0800, Ming Lei wrote: > If .queue_rq() returns BLK_STS_RESOURCE, blk-mq will rerun > the queue in the three situations: > > 1) if BLK_MQ_S_SCHED_RESTART is set > - queue is rerun after one rq is completed, see blk_mq_sched_restart() > which is run from

Re: [PATCH 2/5] dm-mpath: return DM_MAPIO_REQUEUE in case of rq allocation failure

2017-09-15 Thread Bart Van Assche
M_MAPIO_REQUEUE; > } > clone->bio = clone->biotail = NULL; > clone->rq_disk = bdev->bd_disk; So you are reverting the patch below? Thank you very much. commit 1c23484c355ec360ca2f37914f8a4802c6baeead Author: Bart Van Assche <bart.vanass...@wdc.com> D

Re: [PATCH V3 06/12] scsi: sd_zbc: Rearrange code

2017-09-15 Thread Bart Van Assche
On Sat, 2017-09-16 at 07:35 +0900, Damien Le Moal wrote: > rw16 is mandatory for ZBC drives. So it has to be set to true. If the > HBA does not support rw16 (why would that happen ?), then the disk > should not be used. It's good that all HBAs support rw16. But it's nontrivial to analyze whether

Re: [PATCH 1/5] block: don't call blk_mq_delay_run_hw_queue() in case of BLK_STS_RESOURCE

2017-09-18 Thread Bart Van Assche
On Sun, 2017-09-17 at 20:40 +0800, Ming Lei wrote: > "if no request has completed before the delay has expired" can't be a > reason to rerun the queue, because the queue can still be busy. That statement of you shows that there are important aspects of the SCSI core and dm-mpath driver that you

Re: [PATCH v5] blktrace: Fix potentail deadlock between delete & sysfs ops

2017-09-18 Thread Bart Van Assche
On Sat, 2017-09-16 at 19:37 -0700, Waiman Long wrote: > diff --git a/include/linux/fs.h b/include/linux/fs.h > index 339e737..330b572 100644 > --- a/include/linux/fs.h > +++ b/include/linux/fs.h > @@ -448,7 +448,7 @@ struct block_device { > > /* The counter of freeze processes */ >

Re: [RFC PATCH] blk-mq: Fix lost request during timeout

2017-09-18 Thread Bart Van Assche
On Mon, 2017-09-18 at 18:03 -0400, Keith Busch wrote: > I think we've always known it's possible to lose a request during timeout > handling, but just accepted that possibility. It seems to be causing > problems, though, leading to unnecessary error escalation and IO failures. > > The possiblity

Re: [PATCH 1/5] block: don't call blk_mq_delay_run_hw_queue() in case of BLK_STS_RESOURCE

2017-09-19 Thread Bart Van Assche
On Wed, 2017-09-20 at 06:44 +0800, Ming Lei wrote: > For this issue, it isn't same between SCSI and dm-rq. > > We don't need to run queue in .end_io of dm, and the theory is > simple, otherwise it isn't performance issue, and should be I/O hang. > > 1) every dm-rq's request is 1:1 mapped to

[PATCH v2 4/4] scsi-mq: Reduce suspend latency

2017-09-21 Thread Bart Van Assche
Avoid that it can take 200 ms too long to wait for ongoing requests to finish. Note: blk_mq_freeze_queue() uses a wait queue to wait for ongoing requests to finish. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: Mi

[PATCH v2 3/4] block, scsi: Make SCSI device suspend and resume work reliably

2017-09-21 Thread Bart Van Assche
;I/O hangs after resuming from suspend-to-ram" (https://marc.info/?l=linux-block=150340235201348). Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: Ming Lei <ming@redhat.com> Cc: Christoph Hellwig <h...@ls

Re: [PATCH v2 3/4] block, scsi: Make SCSI device suspend and resume work reliably

2017-09-21 Thread Bart Van Assche
On Fri, 2017-09-22 at 06:04 +0800, Ming Lei wrote: > On Thu, Sep 21, 2017 at 02:22:54PM -0700, Bart Van Assche wrote: > > --- a/block/blk-core.c > > +++ b/block/blk-core.c > > @@ -1429,11 +1429,18 @@ struct request *blk_get_request(struct > > reques

Re: [PATCH v2 4/4] scsi-mq: Reduce suspend latency

2017-09-21 Thread Bart Van Assche
On Fri, 2017-09-22 at 07:25 +0800, Ming Lei wrote: > On Thu, Sep 21, 2017 at 10:43:26PM +0000, Bart Van Assche wrote: > > On Fri, 2017-09-22 at 06:06 +0800, Ming Lei wrote: > > > On Thu, Sep 21, 2017 at 02:22:55PM -0700, Bart Van Assche wrote: >

Re: [PATCH v2 4/4] scsi-mq: Reduce suspend latency

2017-09-21 Thread Bart Van Assche
On Fri, 2017-09-22 at 06:06 +0800, Ming Lei wrote: > On Thu, Sep 21, 2017 at 02:22:55PM -0700, Bart Van Assche wrote: > > + } else { > > scsi_run_queue(q); > > + while (atomic_read(>device_busy)) { > > +

Re: [PATCH v2 4/4] scsi-mq: Reduce suspend latency

2017-09-21 Thread Bart Van Assche
On Fri, 2017-09-22 at 07:53 +0800, Ming Lei wrote: > Then that is simply not enough since the issue is that request pool can > be used up easily when SCSI device is quiesced, then no request is left > for RQF_PREEMPT(PM), and I/O hang is triggered. > > It has been discussed for long time, I am

[PATCH v3 5/6] block: Make SCSI device suspend and resume work reliably

2017-09-22 Thread Bart Van Assche
ferences: "I/O hangs after resuming from suspend-to-ram" (https://marc.info/?l=linux-block=150340235201348). Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: Ming Lei <ming@redhat.com> Cc: Christ

[PATCH v3 0/6] Make SCSI device suspend and resume work reliably

2017-09-22 Thread Bart Van Assche
nsider these changes for kernel v4.15. Thanks, Bart. Changes between v2 and v3: - Made md kernel threads freezable. - Changed the approach for quiescing SCSI devices again. - Addressed Ming's review comments. Changes compared to v1 of this patch series: - Changed the approach and rewrote the pat

[PATCH v3 1/6] md: Make md resync and reshape threads freezable

2017-09-22 Thread Bart Van Assche
Some people use the md driver on laptops and use the suspend and resume functionality. Since it is essential that submitting of new I/O requests stops before device quiescing starts, make the md resync and reshape threads freezable. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com&

[PATCH v3 2/6] block: Convert RQF_PREEMPT into REQ_PREEMPT

2017-09-22 Thread Bart Van Assche
that corresponds to this request. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Ming Lei <ming@redhat.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Johannes Thumshirn <jthumsh...@suse.de> --- block/blk-mq-debugfs.c| 2 +-

[PATCH v3 3/6] block: Add the QUEUE_FLAG_PREEMPT_ONLY request queue flag

2017-09-22 Thread Bart Van Assche
This flag will be used in the next patch to let the block layer core know whether or not a SCSI request queue has been quiesced. A quiesced SCSI queue namely only processes RQF_PREEMPT requests. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Ming Lei <ming@redha

[PATCH v3 4/6] scsi: Set QUEUE_FLAG_PREEMPT_ONLY while quiesced

2017-09-22 Thread Bart Van Assche
Make the quiesce state visible to the block layer for the next patch in this series. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: Ming Lei <ming@redhat.com> Cc: Christoph Hellwig <h...@lst.de>

[PATCH v3 6/6] scsi-mq: Reduce suspend latency

2017-09-22 Thread Bart Van Assche
Avoid that it can take 200 ms too long to wait for requests to finish. Note: blk_mq_freeze_queue() uses a wait queue to wait for ongoing requests to finish. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Martin K. Petersen <martin.peter...@oracle.com> Cc: Mi

[PATCH v2 0/4] Make SCSI device suspend and resume work reliably

2017-09-21 Thread Bart Van Assche
nsider these changes for kernel v4.15. Thanks, Bart. Changes compared to v1 of this patch series: - Changed the approach and rewrote the patch series. Bart Van Assche (4): block: Convert RQF_PREEMPT into REQ_PREEMPT block: Add the QUEUE_PREEMPT_ONLY request queue flag block, scsi: Make

[PATCH v2 1/4] block: Convert RQF_PREEMPT into REQ_PREEMPT

2017-09-21 Thread Bart Van Assche
that corresponds to this request. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Ming Lei <ming@redhat.com> Cc: Christoph Hellwig <h...@lst.de> Cc: Hannes Reinecke <h...@suse.com> Cc: Johannes Thumshirn <jthumsh...@suse.de> --- block/blk-mq-debugfs.c| 2 +-

[PATCH v2 2/4] block: Add the QUEUE_PREEMPT_ONLY request queue flag

2017-09-21 Thread Bart Van Assche
This flag will be used in the next patch to let the block layer core know whether or not a SCSI request queue has been quiesced. A quiesced SCSI queue namely only processes RQF_PREEMPT requests. Signed-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Ming Lei <ming@redha

[PATCH] blk-mq: Fix blk_mq_get_request() error path

2017-09-20 Thread Bart Van Assche
ned-off-by: Bart Van Assche <bart.vanass...@wdc.com> Cc: Ming Lei <ming@redhat.com> Cc: <sta...@vger.kernel.org> # v4.13+ --- block/blk-mq.c | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 3f18cff80050..fc2001

Re: [PATCH 2/5] dm-mpath: return DM_MAPIO_REQUEUE in case of rq allocation failure

2017-09-15 Thread Bart Van Assche
On Fri, 2017-09-15 at 16:06 -0400, Mike Snitzer wrote: > The problem is that multipath_clone_and_map() is now treated as common > code (thanks to both blk-mq and old .request_fn now enjoying the use of > blk_get_request) BUT: Ming please understand that this code is used by > old .request_fn too.

Re: [PATCH 5/5] dm-mpath: improve I/O schedule

2017-09-15 Thread Bart Van Assche
On Sat, 2017-09-16 at 00:44 +0800, Ming Lei wrote: > --- > |v4.13+ |v4.13+ > |+scsi_mq_perf |+scsi_mq_perf+patches > - > IOPS(K) |MQ-DEADLINE|MQ-DEADLINE >

Re: [PATCH 5/5] dm-mpath: improve I/O schedule

2017-09-15 Thread Bart Van Assche
On Sat, 2017-09-16 at 00:44 +0800, Ming Lei wrote: > +static void save_path_queue_depth(struct pgpath *p) > +{ > + struct request_queue *q = bdev_get_queue(p->path.dev->bdev); > + > + p->old_nr_requests = q->nr_requests; > + p->queue_depth = q->queue_depth; > + > + /* one extra

Re: [PATCH 5/5] dm-mpath: improve I/O schedule

2017-09-15 Thread Bart Van Assche
On Sat, 2017-09-16 at 00:44 +0800, Ming Lei wrote: > 1) lpfc.lpfc_lun_queue_depth=3, so that it is same with .cmd_per_lun Nobody I know uses such a low queue depth for the lpfc driver. Please also include performance results for a more realistic queue depth. Thanks, Bart.

<    3   4   5   6   7   8   9   10   11   12   >