Re: [Lsf-pc] [LSF/MM TOPIC] block level event logging for storage media management

On Tue 24-01-17 15:18:57, Oleg Drokin wrote: > > On Jan 23, 2017, at 2:27 AM, Dan Williams wrote: > > > [ adding Oleg ] > > > > On Sun, Jan 22, 2017 at 10:00 PM, Song Liu wrote: > >> Hi Dan, > >> > >> I think the the block level event log is more like log only system.

Re: [PATCH] queue stall with blk-mq-sched

On 01/25/2017 08:39 AM, Hannes Reinecke wrote: > On 01/24/2017 11:06 PM, Jens Axboe wrote: >> On 01/24/2017 12:55 PM, Jens Axboe wrote: >>> Try this patch. We only want to bump it for the driver tags, not the >>> scheduler side. >> >> More complete version, this one actually tested. I think this

Re: [PATCHSET v4] blk-mq-scheduling framework

> Il giorno 23 gen 2017, alle ore 18:42, Jens Axboe ha scritto: > > On 01/23/2017 10:04 AM, Paolo Valente wrote: >> >>> Il giorno 18 gen 2017, alle ore 17:21, Jens Axboe ha scritto: >>> >>> On 01/18/2017 08:14 AM, Paolo Valente wrote: according to the

Re: [PATCH 1/6] mmc: block: break out mmc_blk_rw_cmd_abort()

Hi Linus, On 1/24/2017 11:17, Linus Walleij wrote: As a first step toward breaking apart the very complex function mmc_blk_issue_rw_rq() we break out the command abort code. This code assumes "ret" is != 0 and then repeatedly hammers blk_end_request() until the request to the block layer to end

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

On Tue, Jan 24, 2017 at 08:11:52AM +0100, Greg KH wrote: > On Mon, Jan 23, 2017 at 10:52:42AM -0500, Josef Bacik wrote: > > I explained it in the changelog and my response to Wouter. NBD preallocates > > all of its /dev/nbd# devices at modprobe time, so there's no way to add new > > devices as we

Re: [PATCH] queue stall with blk-mq-sched

On 01/25/2017 10:42 AM, Jens Axboe wrote: > On 01/25/2017 10:03 AM, Jens Axboe wrote: >> On 01/25/2017 09:57 AM, Hannes Reinecke wrote: >>> On 01/25/2017 04:52 PM, Jens Axboe wrote: On 01/25/2017 04:10 AM, Hannes Reinecke wrote: >>> [ .. ] > Bah. > > Not quite. I'm still seeing

Re: [PATCH 4/4] nbd: add a nbd-control interface

On Wed, Jan 25, 2017 at 2:47 PM, Josef Bacik wrote: > On Tue, Jan 24, 2017 at 2:11 AM, Greg KH wrote: >> On Mon, Jan 23, 2017 at 10:52:42AM -0500, Josef Bacik wrote: >>> On Mon, Jan 23, 2017 at 10:03 AM, Greg KH >>> Loop

Re: [PATCH] queue stall with blk-mq-sched

On 01/25/2017 09:07 AM, Hannes Reinecke wrote: > On 01/25/2017 08:39 AM, Hannes Reinecke wrote: >> On 01/24/2017 11:06 PM, Jens Axboe wrote: >>> On 01/24/2017 12:55 PM, Jens Axboe wrote: Try this patch. We only want to bump it for the driver tags, not the scheduler side. >>> >>> More

Re: [PATCH 4/4] nbd: add a nbd-control interface

On Tue, Jan 24, 2017 at 2:11 AM, Greg KH wrote: On Mon, Jan 23, 2017 at 10:52:42AM -0500, Josef Bacik wrote: On Mon, Jan 23, 2017 at 10:03 AM, Greg KH wrote: > On Mon, Jan 23, 2017 at 09:57:52AM -0500, Josef Bacik wrote: > > On

Re: [PATCH] queue stall with blk-mq-sched

On 01/25/2017 04:10 AM, Hannes Reinecke wrote: > On 01/25/2017 09:07 AM, Hannes Reinecke wrote: >> On 01/25/2017 08:39 AM, Hannes Reinecke wrote: >>> On 01/24/2017 11:06 PM, Jens Axboe wrote: On 01/24/2017 12:55 PM, Jens Axboe wrote: > Try this patch. We only want to bump it for the

[PATCH v2 04/10] blk-mq: add extra request information to debugfs

From: Omar Sandoval The request pointers by themselves aren't super useful. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- block/blk-mq-debugfs.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git

Re: [PATCH 02/10] blk-mq: add hctx->{state,flags} to debugfs

On Tue, Jan 24, 2017 at 02:25:39PM +0100, Hannes Reinecke wrote: > On 01/23/2017 07:59 PM, Omar Sandoval wrote: > > From: Omar Sandoval > > > > hctx->state could come in handy for bugs where the hardware queue gets > > stuck in the stopped state, and hctx->flags is just useful to

Re: [PATCH 4/4] nbd: add a nbd-control interface

On 1/25/17, 6:23 AM, "arndbergm...@gmail.com on behalf of Arnd Bergmann" wrote: > On Wed, Jan 25, 2017 at 2:47 PM, Josef Bacik wrote: > > On Tue, Jan 24, 2017 at 2:11 AM, Greg KH wrote: > >> On Mon,

[PATCH v2 00/10] blk-mq: move debugging information from sysfs to debugfs

From: Omar Sandoval Changes from v1: - Make the sbitmap seq_file helpers take a (struct sbitmap *) instead of a (void *), since it's not possible to use them directly as the seq_file show helper, anyways - Fix a crash when reading ctx_map because it was attempting to do

[PATCH v2 03/10] blk-mq: move hctx->dispatch and ctx->rq_list from sysfs to debugfs

From: Omar Sandoval These lists are only useful for debugging; they definitely don't belong in sysfs. Putting them in debugfs also removes the limitation of a single page of output. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval ---

[PATCH v2 02/10] blk-mq: add hctx->{state,flags} to debugfs

From: Omar Sandoval hctx->state could come in handy for bugs where the hardware queue gets stuck in the stopped state, and hctx->flags is just useful to know. Signed-off-by: Omar Sandoval --- block/blk-mq-debugfs.c | 42

[PATCH v2 01/10] blk-mq: create debugfs directory tree

From: Omar Sandoval In preparation for putting blk-mq debugging information in debugfs, create a directory tree mirroring the one in sysfs: # tree -d /sys/kernel/debug/block /sys/kernel/debug/block |-- nvme0n1 | `-- mq | |-- 0 | | `-- cpu0

[PATCH v2 05/10] sbitmap: add helpers for dumping to a seq_file

From: Omar Sandoval This is useful debugging information that will be used in the blk-mq debugfs directory. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- include/linux/sbitmap.h | 28 +++ lib/sbitmap.c |

[PATCH v2 09/10] blk-mq: move hctx io_poll, stats, and dispatched from sysfs to debugfs

From: Omar Sandoval These statistics _might_ be useful to userspace, but it's better not to commit to an ABI for these yet. Also, the dispatched file in sysfs couldn't be cleared, so make it clearable like the others in debugfs. Reviewed-by: Hannes Reinecke

[PATCH v2 07/10] blk-mq: move tags and sched_tags info from sysfs to debugfs

From: Omar Sandoval These are very tied to the blk-mq tag implementation, so exposing them to sysfs isn't a great idea. Move the debugging information to debugfs and add basic entries for the number of tags and the number of reserved tags to sysfs. Reviewed-by: Hannes Reinecke

[PATCH v2 06/10] blk-mq: export software queue pending map to debugfs

From: Omar Sandoval This is useful for debugging problems where we've gotten stuck with requests in the software queues. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- block/blk-mq-debugfs.c | 21 + 1 file

[PATCH v2 10/10] blk-mq: move hctx and ctx counters from sysfs to debugfs

From: Omar Sandoval These counters aren't as out-of-place in sysfs as the other stuff, but debugfs is a slightly better home for them. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- block/blk-mq-debugfs.c | 181

[PATCH v2 08/10] blk-mq: add tags and sched_tags bitmaps to debugfs

From: Omar Sandoval These can be used to debug issues like tag leaks and stuck requests. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- block/blk-mq-debugfs.c | 50 ++ 1 file

[PATCH 16/18] block/bsg: move queue creation into bsg_setup_queue

Simply the boilerplate code needed for bsg nodes a bit. Signed-off-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- block/bsg-lib.c | 21 +++-- drivers/scsi/scsi_transport_fc.c| 36

[PATCH 11/18] scsi: remove gfp_flags member in scsi_host_cmd_pool

When using the slab allocator we already decide at cache creation time if an allocation comes from a GFP_DMA pool using the SLAB_CACHE_DMA flag, and there is no point passing the kmalloc-family only GFP_DMA flag to kmem_cache_alloc. Drop all the infrastructure for doing so. Signed-off-by:

[PATCH 14/18] scsi: remove __scsi_alloc_queue

Instead do an internal export of __scsi_init_queue for the transport classes that export BSG nodes. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn --- drivers/scsi/scsi_lib.c | 19

Re: [PATCH] queue stall with blk-mq-sched

On 01/25/2017 10:03 AM, Jens Axboe wrote: > On 01/25/2017 09:57 AM, Hannes Reinecke wrote: >> On 01/25/2017 04:52 PM, Jens Axboe wrote: >>> On 01/25/2017 04:10 AM, Hannes Reinecke wrote: >> [ .. ] Bah. Not quite. I'm still seeing some queues with state 'restart'. I've

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

On Wed, Jan 25, 2017 at 04:48:27PM +, Alex Gartrell wrote: > On 1/25/17, 6:23 AM, "arndbergm...@gmail.com on behalf of Arnd Bergmann" > wrote: > > We have multiple established ways to deal with this kind of problem, the > > most > > common

[PATCH 04/18] block: simplify blk_init_allocated_queue

Return an errno value instead of the passed in queue so that the callers don't have to keep track of two queues, and move the assignment of the request_fn and lock to the caller as passing them as argument doesn't simplify anything. While we're at it also remove two pointless NULL assignments,

[PATCH 03/18] block: fix elevator init check

We can't initalize the elevator fields for flushes as flush share space in struct request with the elevator data. But currently we can't commnicate that a request is a flush through blk_get_request as we can only pass READ or WRITE, and the low-level code looks at the possible NULL bio to check

[PATCH 02/18] md: cleanup bio op / flags handling in raid1_write_request

No need for the local variables, the bio is still live and we can just assigned the bits we want directly. Make me wonder why we can't assign all the bio flags to start with. Signed-off-by: Christoph Hellwig --- drivers/md/raid1.c | 7 ++- 1 file changed, 2 insertions(+), 5

[PATCH 01/18] block: add a op_is_flush helper

This centralizes the checks for bios that needs to be go into the flush state machine. Signed-off-by: Christoph Hellwig --- block/blk-core.c | 8 block/blk-mq-sched.c | 5 ++--- block/blk-mq.c | 4 ++-- drivers/md/bcache/request.c | 2

[PATCH 08/18] scsi_dh_rdac: switch to scsi_execute_req_flags()

From: Hannes Reinecke Switch to scsi_execute_req_flags() and scsi_get_vpd_page() instead of open-coding it. Using scsi_execute_req_flags() will set REQ_QUIET and REQ_PREEMPT, but this is okay as we're evaluating the errors anyway and should be able to send the command even if the

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

> On 25 Jan 2017, at 16:48, Alex Gartrell wrote: > > > If nbd were *all* netlink I think that that'd be fine, but you'd have > problems implementing the NBD_DOIT function in that fashion. So I'd > rather stick to the char device ioctl thing because it's more > consistent

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

On Wed, Jan 25, 2017 at 9:23 AM, Arnd Bergmann wrote: > > They all have their own set of problems, but the needs of nbd as a network > storage interface seem most closely resemble what we have for other network > related interfaces, where we typically use netlink to do the setup,

[PATCH v3 05/10] sbitmap: add helpers for dumping to a seq_file

From: Omar Sandoval This is useful debugging information that will be used in the blk-mq debugfs directory. Reviewed-by: Hannes Reinecke Signed-off-by: Omar Sandoval --- Jens took offense to me making the bitmap dumps binary, so this turns them

Re: [PATCH 04/18] block: simplify blk_init_allocated_queue

> "Christoph" == Christoph Hellwig writes: Christoph> Return an errno value instead of the passed in queue so that Christoph> the callers don't have to keep track of two queues, and move Christoph> the assignment of the request_fn and lock to the caller as Christoph> passing

Re: [PATCH 08/18] scsi_dh_rdac: switch to scsi_execute_req_flags()

> "Christoph" == Christoph Hellwig writes: Christoph> From: Hannes Reinecke Switch to Christoph> scsi_execute_req_flags() and scsi_get_vpd_page() instead of Christoph> open-coding it. Using scsi_execute_req_flags() will set Christoph> REQ_QUIET and REQ_PREEMPT,

Re: [PATCH 09/18] scsi_dh_emc: switch to scsi_execute_req_flags()

> "Christoph" == Christoph Hellwig writes: Christoph> From: Hannes Reinecke Switch to Christoph> scsi_execute_req_flags() and scsi_get_vpd_page() instead of Christoph> open-coding it. Using scsi_execute_req_flags() will set Christoph> REQ_QUIET and REQ_PREEMPT,

Re: [PATCH 14/18] scsi: remove __scsi_alloc_queue

> "Christoph" == Christoph Hellwig writes: Christoph> Instead do an internal export of __scsi_init_queue for the Christoph> transport classes that export BSG nodes. Acked-by: Martin K. Petersen -- Martin K. Petersen Oracle Linux Engineering

Re: [PATCH 15/18] scsi: allocate scsi_cmnd structures as part of struct request

> "Christoph" == Christoph Hellwig writes: Christoph> Rely on the new block layer functionality to allocate Christoph> additional driver specific data behind struct request instead Christoph> of implementing it in SCSI itѕelf. Acked-by: Martin K. Petersen

Re: [PATCH 05/16] dm: always defer request allocation to the owner of the request_queue

On 01/25/17 01:39, Mike Snitzer wrote: > On Tue, Jan 24 2017 at 9:20am -0500, Christoph Hellwig wrote: >> On Tue, Jan 24, 2017 at 05:05:39AM -0500, Mike Snitzer wrote: >>> possible and is welcomed cleanup. The only concern I have is that using >>> get_request() for the old

Re: [PATCH 03/18] block: fix elevator init check

> "Christoph" == Christoph Hellwig writes: Christoph> We can't initalize the elevator fields for flushes as flush Christoph> share space in struct request with the elevator data. But Christoph> currently we can't commnicate that a request is a flush communicate Christoph>

Re: [PATCH 11/18] scsi: remove gfp_flags member in scsi_host_cmd_pool

> "Christoph" == Christoph Hellwig writes: Christoph> When using the slab allocator we already decide at cache Christoph> creation time if an allocation comes from a GFP_DMA pool Christoph> using the SLAB_CACHE_DMA flag, and there is no point passing Christoph> the

Re: [PATCH 10/18] scsi_dh_hp_sw: switch to scsi_execute_req_flags()

> "Christoph" == Christoph Hellwig writes: Christoph> From: Hannes Reinecke Switch to Christoph> scsi_execute_req_flags() instead of using the block interface Christoph> directly. This will set REQ_QUIET and REQ_PREEMPT, but this Christoph> is okay as we're

Re: [PATCH 18/18] block: don't assign cmd_flags in __blk_rq_prep_clone

> "Christoph" == Christoph Hellwig writes: Christoph> These days we have the proper flags set since request Christoph> allocation time. Acked-by: Martin K. Petersen -- Martin K. Petersen Oracle Linux Engineering -- To unsubscribe from this

Re: [PATCH 02/18] md: cleanup bio op / flags handling in raid1_write_request

> "Christoph" == Christoph Hellwig writes: Christoph> No need for the local variables, the bio is still live and we Christoph> can just assigned the bits we want directly. Make me wonder Christoph> why we can't assign all the bio flags to start with. Reviewed-by: Martin K.

Re: [PATCH 01/18] block: add a op_is_flush helper

> "Christoph" == Christoph Hellwig writes: Christoph> This centralizes the checks for bios that needs to be go into Christoph> the flush state machine. Reviewed-by: Martin K. Petersen -- Martin K. Petersen Oracle Linux Engineering -- To

Re: [PATCH 05/18] block: allow specifying size for extra command data

> "Christoph" == Christoph Hellwig writes: Christoph, Christoph> This mirrors the blk-mq capabilities to allocate extra Christoph> drivers-specific data behind struct request by setting a Christoph> cmd_size field, as well as having a constructor / destructor Christoph> for it.

Re: [PATCH 12/18] scsi: respect unchecked_isa_dma for blk-mq

> "Christoph" == Christoph Hellwig writes: Christoph> Currently blk-mq always allocates the sense buffer using Christoph> normal GFP_KERNEL allocation. Refactor the cmd pool code to Christoph> split the cmd and sense allocation and share the code to Christoph> allocate the

Re: [PATCH 13/18] scsi: remove scsi_cmd_dma_pool

> "Christoph" == Christoph Hellwig writes: Christoph> There is no need for GFP_DMA allocations of the scsi_cmnd Christoph> structures themselves, all that might be DMAed to or from is Christoph> the actual payload, or the sense buffers. Acked-by: Martin K. Petersen

[PATCH 13/18] scsi: remove scsi_cmd_dma_pool

There is no need for GFP_DMA allocations of the scsi_cmnd structures themselves, all that might be DMAed to or from is the actual payload, or the sense buffers. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/scsi/scsi.c | 15

[PATCH 06/18] dm: remove incomple BLOCK_PC support

DM tries to copy a few fields around for BLOCK_PC requests, but given that no dm-target ever wires up scsi_cmd_ioctl BLOCK_PC can't actually be sent to dm. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/md/dm-rq.c | 16 1

[PATCH 07/18] dm: always defer request allocation to the owner of the request_queue

DM already calls blk_mq_alloc_request on the request_queue of the underlying device if it is a blk-mq device. But now that we allow drivers to allocate additional data and initialize it ahead of time we need to do the same for all drivers. Doing so and using the new cmd_size infrastructure in

[PATCH 15/18] scsi: allocate scsi_cmnd structures as part of struct request

Rely on the new block layer functionality to allocate additional driver specific data behind struct request instead of implementing it in SCSI itѕelf. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- drivers/scsi/hosts.c | 20 +--

[PATCH 05/18] block: allow specifying size for extra command data

This mirrors the blk-mq capabilities to allocate extra drivers-specific data behind struct request by setting a cmd_size field, as well as having a constructor / destructor for it. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- block/blk-core.c

[PATCH 12/18] scsi: respect unchecked_isa_dma for blk-mq

Currently blk-mq always allocates the sense buffer using normal GFP_KERNEL allocation. Refactor the cmd pool code to split the cmd and sense allocation and share the code to allocate the sense buffers as well as the sense buffer slab caches between the legacy and blk-mq path. Note that this

[PATCH 10/18] scsi_dh_hp_sw: switch to scsi_execute_req_flags()

From: Hannes Reinecke Switch to scsi_execute_req_flags() instead of using the block interface directly. This will set REQ_QUIET and REQ_PREEMPT, but this is okay as we're evaluating the errors anyway and should be able to send the command even if the device is quiesced.

[PATCH 18/18] block: don't assign cmd_flags in __blk_rq_prep_clone

These days we have the proper flags set since request allocation time. Signed-off-by: Christoph Hellwig Reviewed-by: Hannes Reinecke --- block/blk-core.c | 1 - 1 file changed, 1 deletion(-) diff --git a/block/blk-core.c b/block/blk-core.c index 33c5d05e..6bf5ba0

Re: [Lsf-pc] [LSF/MM TOPIC] block level event logging for storage media management

On Jan 25, 2017, at 4:56 AM, Jan Kara wrote: > On Tue 24-01-17 15:18:57, Oleg Drokin wrote: >> >> On Jan 23, 2017, at 2:27 AM, Dan Williams wrote: >> >>> [ adding Oleg ] >>> >>> On Sun, Jan 22, 2017 at 10:00 PM, Song Liu wrote: Hi Dan, I think the the

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

On Wed, Jan 25, 2017 at 06:25:11PM +, Alex Bligh wrote: > > > On 25 Jan 2017, at 16:48, Alex Gartrell wrote: > > > > > > If nbd were *all* netlink I think that that'd be fine, but you'd have > > problems implementing the NBD_DOIT function in that fashion. So I'd > >

Re: [Nbd] [PATCH 4/4] nbd: add a nbd-control interface

On 01/25/2017 03:30 PM, Greg KH wrote: >> Given (because of NBD_DO_IT) we need an ioctl anyway, and we have >> an ioctl that isn't going to go away, it would seem better if possible >> to stick with ioctls, and not introduce either a dependency >> on netlink (which would presumably bloat static

Re: [RFC PATCH v2 0/2] block: fix backing_dev_info lifetime

On Mon, Jan 23, 2017 at 1:17 PM, Thiago Jung Bauermann wrote: > Hello Dan, > > Am Freitag, 6. Januar 2017, 17:02:51 BRST schrieb Dan Williams: >> v1 of these changes [1] was a one line change to bdev_get_queue() to >> prevent a shutdown crash when del_gendisk() races