Re: [dm-devel] [PATCH 01/16] block: add disk/bio-based accounting helpers

2020-05-25 Thread Konstantin Khlebnikov
On 25/05/2020 14.29, Christoph Hellwig wrote: Add two new helpers to simplify I/O accounting for bio based drivers. Currently these drivers use the generic_start_io_acct and generic_end_io_acct helpers which have very cumbersome calling conventions, don't actually return the time they started

Re: [dm-devel] block I/O accounting improvements

2020-05-25 Thread Konstantin Khlebnikov
On 25/05/2020 14.29, Christoph Hellwig wrote: Hi Jens, they series contains various improvement for block I/O accounting. The first bunch of patches switch the bio based drivers to better accounting helpers compared to the current mess. The end contains a fix and various performanc

Re: [dm-devel] [PATCH 05/16] bcache: use bio_{start,end}_io_acct

2020-05-25 Thread Coly Li
On 2020/5/25 19:30, Christoph Hellwig wrote: > Switch bcache to use the nicer bio accounting helpers, and call the > routines where we also sample the start time to give coherent accounting > results. > > Signed-off-by: Christoph Hellwig Acked-by: Coly Li Coly Li > --- >

[dm-devel] [PATCH 10/16] block: move update_io_ticks to blk-core.c

2020-05-25 Thread Christoph Hellwig
All callers are in blk-core.c, so move update_io_ticks over. Signed-off-by: Christoph Hellwig --- block/bio.c | 16 block/blk-core.c | 15 +++ block/blk.h | 1 - 3 files changed, 15 insertions(+), 17 deletions(-) diff --git a/block/bio.c b/block/bio.c

[dm-devel] [PATCH 08/16] zram: nvdimm: use bio_{start, end}_io_acct and disk_{start, end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch zram to use the nicer bio accounting helpers, and as part of that ensure each bio is counted as a single I/O request. Signed-off-by: Christoph Hellwig --- drivers/block/zram/zram_drv.c | 24 ++-- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git

[dm-devel] [PATCH 01/16] block: add disk/bio-based accounting helpers

2020-05-25 Thread Christoph Hellwig
Add two new helpers to simplify I/O accounting for bio based drivers. Currently these drivers use the generic_start_io_acct and generic_end_io_acct helpers which have very cumbersome calling conventions, don't actually return the time they started accounting, and try to deal with accounting for

[dm-devel] [PATCH 12/16] block: account merge of two requests

2020-05-25 Thread Christoph Hellwig
From: Konstantin Khlebnikov Also rename blk_account_io_merge() into blk_account_io_merge_request() to distinguish it from merging request and bio. Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- block/blk-merge.c | 12 +--- 1 file changed, 5

[dm-devel] block I/O accounting improvements

2020-05-25 Thread Christoph Hellwig
Hi Jens, they series contains various improvement for block I/O accounting. The first bunch of patches switch the bio based drivers to better accounting helpers compared to the current mess. The end contains a fix and various performanc improvements. Most of this comes from a series Konstantin

[dm-devel] [PATCH 14/16] block: remove rcu_read_lock() from part_stat_lock()

2020-05-25 Thread Christoph Hellwig
From: Konstantin Khlebnikov The RCU lock is required only in disk_map_sector_rcu() to lookup the partition. After that request holds reference to related hd_struct. Replace get_cpu() with preempt_disable() - returned cpu index is unused. Signed-off-by: Konstantin Khlebnikov [hch: rebased]

[dm-devel] [PATCH 13/16] block: add a blk_account_io_merge_bio helper

2020-05-25 Thread Christoph Hellwig
From: Konstantin Khlebnikov Move the non-"new_io" branch of blk_account_io_start() into separate function. Fix merge accounting for discards (they were counted as write merges). The new blk_account_io_merge_bio() doesn't call update_io_ticks() unlike blk_account_io_start(), as there is no

[dm-devel] [PATCH 11/16] block: always use a percpu variable for disk stats

2020-05-25 Thread Christoph Hellwig
percpu variables have a perfectly fine working stub implementation for UP kernels, so use that. Signed-off-by: Christoph Hellwig --- block/blk.h | 2 +- block/genhd.c | 12 +++-- block/partitions/core.c | 5 ++-- include/linux/genhd.h | 13 -

[dm-devel] [PATCH 02/16] drbd: use bio_{start,end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch drbd to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig --- drivers/block/drbd/drbd_req.c | 27 --- 1 file changed, 4 insertions(+), 23 deletions(-) diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c index

[dm-devel] [PATCH 05/16] bcache: use bio_{start,end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch bcache to use the nicer bio accounting helpers, and call the routines where we also sample the start time to give coherent accounting results. Signed-off-by: Christoph Hellwig --- drivers/md/bcache/request.c | 18 -- 1 file changed, 4 insertions(+), 14 deletions(-) diff

[dm-devel] [PATCH 07/16] nvdimm: use bio_{start,end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig --- drivers/nvdimm/blk.c | 6 -- drivers/nvdimm/btt.c | 6 -- drivers/nvdimm/nd.h | 19 --- drivers/nvdimm/pmem.c | 6 -- 4 files changed, 12 insertions(+), 25 deletions(-)

[dm-devel] [PATCH 15/16] block: use __this_cpu_add() instead of access by smp_processor_id()

2020-05-25 Thread Christoph Hellwig
From: Konstantin Khlebnikov Most architectures have fast path to access percpu for current cpu. The required preempt_disable() is provided by part_stat_lock(). Signed-off-by: Konstantin Khlebnikov [hch: rebased] Signed-off-by: Christoph Hellwig --- include/linux/part_stat.h | 2 +- 1 file

[dm-devel] [PATCH 09/16] block: remove generic_{start, end}_io_acct

2020-05-25 Thread Christoph Hellwig
Remove these now unused functions. Signed-off-by: Christoph Hellwig --- block/bio.c | 39 --- include/linux/bio.h | 6 -- 2 files changed, 45 deletions(-) diff --git a/block/bio.c b/block/bio.c index 9c101a0572ca2..3e89c7b37855a 100644 ---

[dm-devel] [PATCH 04/16] lightnvm/pblk: use bio_{start, end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch rsxx to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig --- drivers/lightnvm/pblk-cache.c | 8 +++- drivers/lightnvm/pblk-read.c | 11 --- 2 files changed, 7 insertions(+), 12 deletions(-) diff --git a/drivers/lightnvm/pblk-cache.c

[dm-devel] [PATCH 06/16] dm: use bio_{start,end}_io_acct

2020-05-25 Thread Christoph Hellwig
Switch dm to use the nicer bio accounting helpers. Signed-off-by: Christoph Hellwig --- drivers/md/dm.c | 9 ++--- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/md/dm.c b/drivers/md/dm.c index f215b86664484..3f39fa1ac756e 100644 --- a/drivers/md/dm.c +++

Re: [dm-devel] [PATCH] block: Improve io_opt limit stacking

2020-05-25 Thread Keith Busch
On Fri, May 22, 2020 at 09:36:18AM -0400, Martin K. Petersen wrote: > > >>> + if (t->io_opt & (t->physical_block_size - 1)) > >>> + t->io_opt = lcm(t->io_opt, t->physical_block_size); > > > >> Any comment on this patch ? Note: the patch the patch "nvme: Fix > >> io_opt limit setting" is

Re: [dm-devel] [PATCH 10/12] dm-zoned: support arbitrary number of devices

2020-05-25 Thread Damien Le Moal
On 2020/05/25 16:52, Hannes Reinecke wrote: > On 5/25/20 4:36 AM, Damien Le Moal wrote: >> On 2020/05/23 0:39, Hannes Reinecke wrote: >>> Remove the hard-coded limit of two devices and support an unlimited >>> number of additional zoned devices. >>> With that we need to increase the device-mapper

Re: [dm-devel] [PATCH 11/12] dm-zoned: round-robin load balancer for reclaiming zones

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:42 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: When reclaiming zones we should arbitrate between the zoned devices to get a better throughput. So implement a simple round-robin load balancer between the zoned devices. Signed-off-by: Hannes Reinecke ---

Re: [dm-devel] [PATCH 10/12] dm-zoned: support arbitrary number of devices

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:36 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: Remove the hard-coded limit of two devices and support an unlimited number of additional zoned devices. With that we need to increase the device-mapper version number to 3.0.0 as we've modified the interface.

Re: [dm-devel] [PATCH 08/12] dm-zoned: move random and sequential zones into struct dmz_dev

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:27 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: Random and sequential zones should be part of the respective device structure to make arbitration between devices possible. Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 143

Re: [dm-devel] [PATCH 05/12] dm-zoned: add device pointer to struct dm_zone

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:15 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: Add a pointer to the containing device to struct dm_zone and kill dmz_zone_to_dev(). Signed-off-by: Hannes Reinecke --- drivers/md/dm-zoned-metadata.c | 47 --

Re: [dm-devel] [PATCH 03/12] dm-zoned: use on-stack superblock for tertiary devices

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:09 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: Checking the teriary superblock just consists of validating UUIDs, s/teriary/tertiary crcs, and the generation number; it doesn't have contents which would be required during the actual operation. So we

Re: [dm-devel] [PATCH 02/12] dm-zoned: convert to xarray

2020-05-25 Thread Hannes Reinecke
On 5/25/20 4:01 AM, Damien Le Moal wrote: On 2020/05/23 0:39, Hannes Reinecke wrote: The zones array is getting really large, and large arrays tend to wreak havoc with the caches. s/caches/CPU cache, may be ? So convert it to xarray to become more cache friendly. Signed-off-by: Hannes