Re: [PATCH 04/10] block: Move bdi_unregister() to del_gendisk()

2017-02-09 Thread NeilBrown
by Omar [1]. Moving > bdi_unregister() to del_gendisk() fixes these problems as well since > del_gendisk() gets called from sd_remove() before freeing the device > number. > > This also makes device_add_disk() (calling bdi_register_owner()) more > symmetric with del_gendisk(). What an

Re: [PATCH 07/10] writeback: Implement reliable switching to default writeback structure

2017-02-09 Thread NeilBrown
__I_WB_SWITCH); I note that __inode_wait_for_state_bit() can drop and reclaim ->i_lock. is it possible that: !inode_to_wb_is_valid(inode) || inode_to_wb(inode) == >wb) could change while ->i_lock is unlocked? It would be particular unfortunate if inode_to_wb(inode) became >wb due to

Re: [Lsf-pc] [LSF/MM TOPIC] do we really need PG_error at all?

2017-02-28 Thread NeilBrown
On Mon, Feb 27 2017, Jeff Layton wrote: > On Tue, 2017-02-28 at 10:32 +1100, NeilBrown wrote: >> On Mon, Feb 27 2017, Andreas Dilger wrote: >> >> > >> > My thought is that PG_error is definitely useful for applications to get >> > correct errors ba

Re: [LSF/MM TOPIC] do we really need PG_error at all?

2017-02-26 Thread NeilBrown
nd that no finer granularity could be available. I think the question here is: Do filesystems need the pagecache to record which pages have seen an IO error? I think that for write errors, there is no value in recording block-oriented error status - only file-oriented status. For read errors, it might

Re: [LSF/MM TOPIC] do we really need PG_error at all?

2017-02-26 Thread NeilBrown
On Sun, Feb 26 2017, James Bottomley wrote: > On Mon, 2017-02-27 at 08:03 +1100, NeilBrown wrote: >> On Sun, Feb 26 2017, James Bottomley wrote: >> >> > [added linux-scsi and linux-block because this is part of our error >> > handling as well] >> > On S

Re: blockdev kernel regression (bugzilla 173031)

2016-10-05 Thread NeilBrown
On Wed, Oct 05 2016, Francesco Dolcini wrote: > Hi all, > Adding linux-block in CC for this bug report. > > > On Wed, Oct 05, 2016 at 01:00:57PM +1100, NeilBrown wrote: >> On Wed, Oct 05 2016, Francesco Dolcini wrote: >> > Hi all, >> > moving from k

Re: blockdev kernel regression (bugzilla 173031)

2016-10-06 Thread NeilBrown
On Thu, Oct 06 2016, Francesco Dolcini wrote: > On Thu, Oct 06, 2016 at 04:42:52PM +1100, NeilBrown wrote: > cc >> Maybe there is a race, but that seems unlikely. > > Consider that just hot removal while writing is not enough to > reproduce systematically the bug. >

Re: [PATCH] badblocks: fix overlapping check for clearing

2016-10-06 Thread NeilBrown
;= 0 && > -BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) { > +(BB_OFFSET(p[lo]) + BB_LEN(p[lo]) > s) && > +(BB_OFFSET(p[lo]) <= target)) { Ditto. But the code is, I think, correct. Just not how I would have written it.

Re: [PATCH] badblocks: fix overlapping check for clearing

2016-10-18 Thread NeilBrown
> int ack = BB_ACK(p[lo]); >> >> sector_t a = BB_OFFSET(p[lo]); >> >> @@ -377,7 +378,8 @@ int badblocks_clear(struct badblocks *bb, sector_t s, >> >> int sectors) >> >> lo--; >

[PATCH] block_dev: don't test bdev->bd_contains when it is not stable.

2016-11-24 Thread NeilBrown
implement bd_claiming and claiming block") Cc: sta...@vger.kernel.org (v2.6.35+) Signed-off-by: NeilBrown <ne...@suse.com> --- fs/block_dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 05b553368bb4..9166b9f63d33 100644 --- a/fs/blo

[md PATCH 1/6] md/failfast: add failfast flag for md to be used by some personalities.

2016-11-17 Thread NeilBrown
with the correct data. It is expected that if transient errors, such as cable unplug, are possible, then something in user-space will revalidate failed devices and re-add them when they appear to be working again. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/md.c

[md PATCH 2/6] md: Use REQ_FAILFAST_* on metadata writes where appropriate

2016-11-17 Thread NeilBrown
a failure writing metadata but the device doesn't fail, it must be the last device so we re-write without FAILFAST to improve chance of success. We also flag the device as LastDev so that future metadata updates don't waste time on failfast writes. Signed-off-by: NeilBrown <ne...@suse.com> --- driv

[md PATCH 5/6] md/raid10: add failfast handling for reads.

2016-11-17 Thread NeilBrown
. if there are > 2 devices. If we get a failure we will fail the device and complete the resync/recovery with remaining devices. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/raid10.c | 49 - drivers/md/raid10.h |2 ++

[md PATCH 6/6] md/raid10: add failfast handling for writes.

2016-11-17 Thread NeilBrown
. If a normal write fails, but the device cannot be marked Faulty (must be only one left), we queue for write error handling which calls narrow_write_error() to write the block synchronously without any failfast flags. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/raid10.c

[md PATCH 4/6] md/raid1: add failfast handling for writes.

2016-11-17 Thread NeilBrown
. If a normal failfast write fails, but the device cannot be failed (must be only one left), we queue for write error handling. This will call narrow_write_error() to retry the write synchronously and without any FAILFAST flags. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/raid1.c

[PATCH/RFC] add "failfast" support for raid1/raid10.

2016-11-17 Thread NeilBrown
on all drives. Otherwise the timeout would drift towards (say) 10 times the typical response time. So: comments most welcome. As I say, this does address a genuine need. Just find it hard to like it :-( Thanks, NeilBrown --- NeilBrown (6): md/failfast: add failfast flag

[PATCH] block: call trace_block_split() from bio_split()

2016-11-17 Thread NeilBrown
were traced before will still be traced. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c | 1 + block/blk-merge.c | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/block/bio.c b/block/bio.c index db85c5753a76..212ea95a7401 100644 --- a/block/bio.c +++ b/block/bi

[md PATCH 3/6] md/raid1: add failfast handling for reads.

2016-11-17 Thread NeilBrown
_pending == 0. The spinlocked region in raid1_error() in enlarged to ensure that if two bios, reading from two different devices, fail at the same time, then there is no risk that both devices will be marked faulty, leaving zero "In_sync" devices. Signed-off-by: NeilBrown <ne...@suse.com> -

Re: "creative" bio usage in the RAID code

2016-11-13 Thread NeilBrown
that is bad. So we repeat the read with finer granularity (pages in the current code, though device block would be ideal) and only recovery bad blocks for individual pages which are bad and cannot be fixed. NeilBrown signature.asc Description: PGP signature

Re: "creative" bio usage in the RAID code

2016-11-13 Thread NeilBrown
g_back_size; (any driver that needs those, would see a 'request' rather than a 'bio' and so could use rq->special) raid5.c could then use bi_dev_private (or bi_special, or whatever it is call). NeilBrown signature.asc Description: PGP signature

[PATCH - v2] block: call trace_block_split() from bio_split()

2016-11-20 Thread NeilBrown
ed before will still be traced. Reviewed-by: Christoph Hellwig <h...@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- Thanks Christoph. This adds the wrap and the reviewed-by. NeilBrown block/bio.c | 2 ++ block/blk-merge.c | 1 - 2 files changed, 2 insertions(+), 1 del

Re: raid0 vs. mkfs

2016-12-11 Thread NeilBrown
d issue a Why? Why cannot you just split of the start of the bio and chain it with the rest of the bio? If the bio doesn't start at the beginning of a stripe, just split of the first (partial) chunk exactly was we currently do. If it does start at the beginning of a stripe, then split off a whole number of stripes and allocate one bio for each device. Chain those to the original bio together with any remainder (which isn't a whole stripe). I think that if you make use of bio_split() and bio_chain() properly, the code will be much simpler. NeilBrown signature.asc Description: PGP signature

[PATCH v2] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
bio_endio_notrace(). This patch only adds uses of this in core code. Separate patches will be needed to update the filesystems to avoid tracing. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio-integrity.c | 4 ++-- block/bio.c

Re: [PATCH] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
On Wed, Mar 22 2017, Christoph Hellwig wrote: > On Wed, Mar 22, 2017 at 01:38:09PM +1100, NeilBrown wrote: >> >> Currently only dm and md/raid5 bios trigger trace_block_bio_complete(). >> Now that we have bio_chain(), it is not possible, in general, for a >> driver to

[PATCH v3] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
() is called (which generates a QUEUED event), then bi_endio() will create a single COMPLETE event for each range that the bio is split into, unless the driver has explicitly requested it not to. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c | 13 + blo

Re: [PATCH v2] block: trace completion of all bios.

2017-03-23 Thread NeilBrown
On Thu, Mar 23 2017, Ming Lei wrote: > On Thu, Mar 23, 2017 at 05:29:02PM +1100, NeilBrown wrote: >> >> /** >> + * bio_endio_notrace - end I/O on a bio without tracing >> + * @bio:bio >> + * >> + * Description: >> + * bio_

Re: [PATCH v3 02/14] md: move two macros into md.h

2017-03-23 Thread NeilBrown
le. Possibly we should find more meaningful numbers, or make them auto-size or something. I'm also happy for them to stay as they are for now. But I don't think we should pretend that they are meaningful. Thanks, NeilBrown > > Signed-off-by: Ming Lei <tom.leim...@gmail.com> >

Re: [PATCH v3 03/14] md: prepare for managing resync I/O pages in clean way

2017-03-24 Thread NeilBrown
> Signed-off-by: Ming Lei <tom.leim...@gmail.com> > --- > drivers/md/md.h | 50 ++ > 1 file changed, 50 insertions(+) I don't think this should go in md.h Maybe create a "raid1-10.h" or similar if you really want t

Re: [PATCH v3] block: trace completion of all bios.

2017-03-26 Thread NeilBrown
On Fri, Mar 24 2017, Ming Lei wrote: > On Fri, Mar 24, 2017 at 8:07 AM, NeilBrown <ne...@suse.com> wrote: ... >> @@ -102,6 +102,8 @@ struct bio { >> #define BIO_REFFED 8 /* bio has elevated ->bi_cnt */ >> #define BIO_THROTTLED 9 /* This

Re: [PATCH v3 02/14] md: move two macros into md.h

2017-03-27 Thread NeilBrown
; us to keep the raid1.c code around instead of making it a special short > cut case in raid10.c? Patches welcome. They would need to handle write-mostly and write-behind. They would also need to avoid the assumption of a chunk size for RAID1. Undoubtedly do-able. Hard to say how beneficial it would

Re: [PATCH v3] block: trace completion of all bios.

2017-03-27 Thread NeilBrown
code) calls it. Thanks, NeilBrown signature.asc Description: PGP signature

Re: [dm-devel] [PATCH v3] block: trace completion of all bios.

2017-03-27 Thread NeilBrown
On Mon, Mar 27 2017, Christoph Hellwig wrote: > On Mon, Mar 27, 2017 at 08:49:57PM +1100, NeilBrown wrote: >> On Mon, Mar 27 2017, Christoph Hellwig wrote: >> >> > I don't really like the flag at all. I'd much prefer a __bio_endio >> > with a 'bool trace' fla

[PATCH] block: trace completion of all bios.

2017-03-21 Thread NeilBrown
trace_block_bio_complete() pairs with trace_block_bio_queue(). Any bio for which a 'queue' event is traced, will subsequently generate a 'complete' event. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c| 3 +++ drivers/md/dm.c| 1 - drivers/md/raid5.c | 8 3 files chan

Re: [RFC PATCH] blk: reset 'bi_next' when bio is done inside request

2017-04-04 Thread NeilBrown
On Tue, Apr 04 2017, Michael Wang wrote: > On 04/04/2017 11:37 AM, NeilBrown wrote: >> On Tue, Apr 04 2017, Michael Wang wrote: > [snip] >>>> >>>> If sync_request_write() is using a bio that has already been used, it >>>> should call bio_reset() and

Re: [PATCH] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-04 Thread NeilBrown
On Tue, Apr 04 2017, Ming Lei wrote: > On Mon, Apr 3, 2017 at 9:18 AM, NeilBrown <ne...@suse.com> wrote: >> >> When a filesystem is mounted from a loop device, writes are >> throttled by balance_dirty_pages() twice: once when writing >> to the filesystem and once

[PATCH v2] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-04 Thread NeilBrown
; Signed-off-by: NeilBrown <ne...@suse.com> --- I moved where the flag is set, thanks to suggestion from Ming Lei. I've preserved the *-by: tags I was offered despite the code being different, as the concept is identical. Thanks, NeilBrown drivers/block/loop.c | 1 + 1 file changed, 1 insert

Re: [PATCH] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-04 Thread NeilBrown
support that :-( I guess it might be time to upgraded. It seems that there is not "mount -o direct_loop" or similar, so you have to do the losetup and the mount separately. Any thoughts on whether that should be changed ? Thanks, NeilBrown signature.asc Description: PGP signature

Re: [RFC PATCH] blk: reset 'bi_next' when bio is done inside request

2017-04-04 Thread NeilBrown
On Tue, Apr 04 2017, Michael Wang wrote: > Hi, Neil > > On 04/03/2017 11:25 PM, NeilBrown wrote: >> On Mon, Apr 03 2017, Michael Wang wrote: >> >>> blk_attempt_plug_merge() try to merge bio into request and chain them >>> by 'bi_next', while after th

Re: [RFC PATCH] blk: reset 'bi_next' when bio is done inside request

2017-04-03 Thread NeilBrown
er I don't see how that would happen. Can you give specific details on the situation that triggers the bug? Thanks, NeilBrown > BUG_ON(bio->bi_next) > > After reset the 'bi_next' this can no longer happen. > > Signed-off-by: Michael Wang <yun.w...@profitbricks.com> &g

Re: [PATCH v2] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-05 Thread NeilBrown
On Wed, Apr 05 2017, Michal Hocko wrote: > On Wed 05-04-17 09:19:27, Michal Hocko wrote: >> On Wed 05-04-17 14:33:50, NeilBrown wrote: > [...] >> > diff --git a/drivers/block/loop.c b/drivers/block/loop.c >> > index 0ecb6461ed81..44b3506fd086 100644 >> >

[PATCH 00/11] block: assorted cleanup for bio splitting and cloning.

2017-04-20 Thread NeilBrown
, bio_clone_fast() does. Finally we remove bio_clone() (which is no longer used) and stop blk_bio_segment_split() from splitting after BIO_MAX_PAGES, as that is no longer necessary. --- NeilBrown (11): blk: remove bio_set arg from blk_queue_split() blk: make the bioset rescue_workqueue optional

[PATCH 08/11] xen-blkfront: remove bio splitting.

2017-04-20 Thread NeilBrown
bios that are re-submitted will pass through blk_queue_split() when blk_queue_bio() is called, and this will split the bio if necessary. There is no longer any need to do this splitting in xen-blkfront. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/xen-blkfront.c

[PATCH 09/11] bcache: use kmalloc to allocate bio in bch_data_verify()

2017-04-19 Thread NeilBrown
is used either for the bio or the pages. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/bcache/debug.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/md/bcache/debug.c b/drivers/md/bcache/debug.c index 06f55056aaae..35a5a7210e51 100644 --- a/driv

[PATCH 11/11] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-04-19 Thread NeilBrown
, and bio_clone_fast() doesn't clone the bi_io_vec. The main user of of bio_clone_bioset() at this level is bounce.c, and bouncing now happens before blk_bio_segment_split(), so that is not of concern. So remove the big helpful comment and the code. Signed-off-by: NeilBrown <ne...@suse.com> --- blo

[PATCH 06/11] drbd: use bio_clone_fast() instead of bio_clone()

2017-04-19 Thread NeilBrown
, but it is generally best to avoid sharing bio_sets unless you can be certain that there are no interdependencies. So create a new bio_set, drbd_io_bio_set, and use bio_clone_fast(). Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/drbd/drbd_int.h |3 +++ drivers/block/drbd/drbd_

[PATCH 07/11] pktcdvd: use bio_clone_fast() instead of bio_clone()

2017-04-19 Thread NeilBrown
, pkt_bio_set, can also be use for the bio_split() call as the two allocations (bio_clone_fast, and bio_split) are independent, neither can block a bio allocated by the other. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/pktcdvd.c | 12 ++-- 1 file changed, 10 insertions

[PATCH 04/11] block: Improvements to bounce-buffer handling

2017-04-19 Thread NeilBrown
could lead to a deadlock. So: - allocate 2 private biosets for blk_queue_bounce, one for splitting enormous bios and one for cloning bios. - add code to split a bio that exceeds BIO_MAX_PAGES. - Fix up the comments in blk_queue_split() Signed-off-by: NeilBrown <ne...@suse.com> --- block/bl

[PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-04-19 Thread NeilBrown
-by: NeilBrown <ne...@suse.com> --- block/bio.c | 28 block/blk-core.c |2 +- drivers/md/bcache/super.c |4 ++-- drivers/md/dm-crypt.c |2 +- drivers/md/dm-io.c|2 +- drivers/md/dm.c |5 +++-- include

[PATCH 03/11] blk: use non-rescuing bioset for q->bio_split.

2017-04-19 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- 1 file changed, 1

[PATCH 01/11] blk: remove bio_set arg from blk_queue_split()

2017-04-19 Thread NeilBrown
it inside blk_queue_split() Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- block/blk-merge.c |9 - block/blk-mq.c|2 +- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/

Re: [PATCH 02/11] blk: make the bioset rescue_workqueue optional.

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Christoph Hellwig wrote: > On Thu, Apr 20, 2017 at 03:38:48PM +1000, NeilBrown wrote: >> This patch converts bioset_create() and >> bioset_create_nobvec() to not create a workqueue so >> alloctions will never trigger punt_bios_to_rescuer().

Re: [PATCH 11/11] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Ming Lei wrote: > On Fri, Apr 21, 2017 at 7:34 PM, Christoph Hellwig <h...@infradead.org> wrote: >> On Thu, Apr 20, 2017 at 03:38:50PM +1000, NeilBrown wrote: >>> blk_bio_segment_split() makes sure bios have no more than >>> BIO_M

Re: [PATCH 01/11] blk: remove bio_set arg from blk_queue_split()

2017-04-23 Thread NeilBrown
On Sat, Apr 22 2017, Javier González wrote: >> On 20 Apr 2017, at 07.38, NeilBrown <ne...@suse.com> wrote: >> >> blk_queue_split() is always called with the last arg being q->bio_split, >> where 'q' is the first arg. >> >> Also blk_queue_split() so

Re: [PATCH 11/11] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-04-23 Thread NeilBrown
On Fri, Apr 21 2017, Christoph Hellwig wrote: > On Thu, Apr 20, 2017 at 03:38:50PM +1000, NeilBrown wrote: >> blk_bio_segment_split() makes sure bios have no more than >> BIO_MAX_PAGES entries in the bi_io_vec. >> This was done because bio_clone_bioset() (when given a >

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
rom some easy-to-fix places. The series summary is below. NeilBrown NeilBrown (5): blk: improve order of bio handling in generic_make_request() blk: remove bio_set arg from blk_queue_split() blk: make the bioset rescue_workqueue optional. blk: use non-rescuing bioset fo

[PATCH 1/5 v3] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
ray of 2 lists, and relevant code examines both of them. Ref: http://www.spinics.net/lists/raid/msg54680.html Tested-by: Jinpu Wang <jinpu.w...@profitbricks.com> Inspired-by: Lars Ellenberg <lars.ellenb...@linbit.com> Signed-off-by: NeilBrown <ne...@suse.com> --- block

[PATCH 2/5] blk: remove bio_set arg from blk_queue_split()

2017-03-09 Thread NeilBrown
it inside blk_queue_split() Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- block/blk-merge.c |7 +++ block/blk-mq.c|4 ++-- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/

[PATCH 3/5] blk: make the bioset rescue_workqueue optional.

2017-03-09 Thread NeilBrown
() and bioset_create_nobvec() are converted to the _rescued() version, so that not change in behaviour is experienced. It is hoped that most, if not all, bioset can end up being the non-rescued version. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c

[PATCH 4/5] blk: use non-rescuing bioset for q->bio_split.

2017-03-09 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be and other bios allocated fro q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- 1 file changed, 1

[PATCH 5/5] block_dev: make blkdev_dio_pool a non-rescuing bioset

2017-03-09 Thread NeilBrown
Allocations from blkdev_dio_pool are never made under generic_make_request, so this bioset does not need a rescuer thread. Signed-off-by: NeilBrown <ne...@suse.com> --- fs/block_dev.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
On Thu, Mar 09 2017, Jens Axboe wrote: > On 03/09/2017 09:32 PM, NeilBrown wrote: >> >> I started looking further at the improvements we can make once >> generic_make_request is fixed, and realised that I had missed an >> important detail in this patch. >> Seve

[PATCH 3/5] blk: make the bioset rescue_workqueue optional.

2017-03-09 Thread NeilBrown
() and bioset_create_nobvec() are converted to the _rescued() version, so that no change in behaviour is experienced. It is hoped that most, if not all, biosets can end up being the non-rescued version. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c

[PATCH 4/5] blk: use non-rescuing bioset for q->bio_split.

2017-03-09 Thread NeilBrown
in, as it might still be too big to handle directly. By this time there cannot be any other bios allocated from q->bio_split in the generic_make_request() queue. So no rescuing will ever be needed. Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- 1 file changed, 1

[PATCH 0/5] Updates following recent generic_make_request improvement

2017-03-09 Thread NeilBrown
. It is possible that the changes to btrfs and xfs can just be dropped as a subsequent patch will be needed to revert them anyway. They are there only to be able to say that "blk: make the bioset rescue_workqueue optional." doesn't change any functionality at all. Thanks, NeilBrown --- Ne

[PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread NeilBrown
() recovery thread for many block devices, and eventually it may be possible to remove it completely. Ref: http://www.spinics.net/lists/raid/msg54680.html Tested-by: Jinpu Wang <jinpu.w...@profitbricks.com> Inspired-by: Lars Ellenberg <lars.ellenb...@linbit.com> Signed-off-by: NeilBrown <

Re: [dm-devel] [PATCH 0/5] Updates following recent generic_make_request improvement

2017-03-12 Thread NeilBrown
On Sat, Mar 11 2017, Jens Axboe wrote: > On 03/09/2017 11:00 PM, NeilBrown wrote: >> This is a rebase of the series I sent earlier, based on the >> very latest from Linus, which included my first patch. >> >> The first fixes a problem that patch introduced, and so sho

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread NeilBrown
dent. I actually now think that change probably wasn't necessary. It is safer though. It ensures that current functionality isn't removed without a clear justification. Thanks, NeilBrown signature.asc Description: PGP signature

[PATCH] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-02 Thread NeilBrown
become stable again, though not as fast as the no-loop-back case: 53-72 seconds. There may be room for further improvement as the total overhead still seems too high, but this is a big improvement. Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/loop.c | 3 +++ 1 file chan

[PATCH v3] loop: Add PF_LESS_THROTTLE to block/loop device thread.

2017-04-06 Thread NeilBrown
t; Suggested-by: Michal Hocko <mho...@suse.com> Acked-by: Michal Hocko <mho...@suse.com> Signed-off-by: NeilBrown <ne...@suse.com> --- Hi Jens, I think this version meets with everyone's approval. Thanks, NeilBrown drivers/block/loop.c | 8 +++- 1 file changed, 7 insertions

[PATCH 0/2] Trace completion of all bios

2017-04-06 Thread NeilBrown
Hi Jens, I think all objections to this patch have been answered so I'm resending. I've added a small cleanup patch first which makes some small enhancements to the documentation and #defines for the bio->flags field. Thanks, NeilBrown --- NeilBrown (2): block: simple improveme

[PATCH 2/2] block: trace completion of all bios.

2017-04-06 Thread NeilBrown
() is called (which generates a QUEUED event), then bi_endio() will create a single COMPLETE event for each range that the bio is split into, unless the driver has explicitly requested it not to. Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c | 13 + blo

[PATCH 1/2] block: simple improvements for bio->flags

2017-04-06 Thread NeilBrown
arger than needed. The BVEC_POOL_IDX() ranges from 0 to 6, so 3 bits are sufficient. This patch make improvements in each of these areas. Signed-off-by: NeilBrown <ne...@suse.com> --- include/linux/blk_types.h | 22 +- 1 file changed, 13 insertions(+), 9 deletion

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-13 Thread NeilBrown
On Thu, Jul 13 2017, Shaohua Li wrote: > On Thu, Jul 13, 2017 at 05:20:52PM +0800, Ming Lei wrote: >> On Thu, Jul 13, 2017 at 01:09:28PM +1000, NeilBrown wrote: >> > On Thu, Jul 13 2017, Ming Lei wrote: >> > >> > > On Thu, Jul 13, 2017 at 10:01:33AM +1000,

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-09 Thread NeilBrown
gt;idx = 0; This is the only place the ->idx is initialized, in r1buf_pool_alloc(). The mempool alloc function is suppose to allocate memory, not initialize it. If the mempool_alloc() call cannot allocate memory it will use memory from the pool. If this memory has already been used, then it will no longer have the initialized value. In short: you need to initialise memory *after* calling mempool_alloc(), unless you ensure it is reset to the init values before calling mempool_free(). https://bugzilla.kernel.org/show_bug.cgi?id=196307 NeilBrown signature.asc Description: PGP signature

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-10 Thread NeilBrown
On Mon, Jul 10 2017, Shaohua Li wrote: > On Mon, Jul 10, 2017 at 03:25:41PM +0800, Ming Lei wrote: >> On Mon, Jul 10, 2017 at 02:38:19PM +1000, NeilBrown wrote: >> > On Mon, Jul 10 2017, Ming Lei wrote: >> > >> > > On Mon, Jul 10, 2017 at 11:35:12AM +0800,

Re: [PATCH v3 05/14] md: raid1: don't use bio's vec table to manage resync pages

2017-07-09 Thread NeilBrown
On Mon, Jul 10 2017, Ming Lei wrote: > On Mon, Jul 10, 2017 at 11:35:12AM +0800, Ming Lei wrote: >> On Mon, Jul 10, 2017 at 7:09 AM, NeilBrown <ne...@suse.com> wrote: ... >> >> + >> >> + rp->idx = 0; >> > >> > This is

Re: [PATCH 1/2] md: remove 'idx' from 'struct resync_pages'

2017-07-12 Thread NeilBrown
y > responsible for allocation, we can't use that for initialization. > > Suggested-by: NeilBrown <ne...@suse.com> > Reported-by: NeilBrown <ne...@suse.com> > Fixes: f0250618361d(md: raid10: don't use bio's vec table to manage resync > pages) > Fixes: 98d30c5812c

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-12 Thread NeilBrown
On Thu, Jul 13 2017, Ming Lei wrote: > On Thu, Jul 13, 2017 at 10:01:33AM +1000, NeilBrown wrote: >> On Wed, Jul 12 2017, Ming Lei wrote: >> >> > We will support multipage bvec soon, so initialize bvec >> > table using the standardy way instead of writing t

Re: [PATCH 2/2] md: raid1/raid10: initialize bvec table via bio_add_page()

2017-07-12 Thread NeilBrown
't belong here. If you want to remove code duplication, then work on moving all raid1 functionality into raid10.c, then discard raid1.c Or at the very least, have a separate "raid1-10.c" file for the common code. NeilBrown > + > #ifndef MODULE > > /* > diff --

Re: [PATCH v2 3/3] md: raid1-10: move raid1/raid10 common code into raid1-10.c

2017-07-16 Thread NeilBrown
On Fri, Jul 14 2017, Ming Lei wrote: > No function change, just move 'struct resync_pages' and related > helpers into raid1-10.c > > Signed-off-by: Ming Lei <ming@redhat.com> Thanks for doing this! I'm quite happy with this approach - and with all patches in this series.

Re: [PATCH 13/13] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-05-02 Thread NeilBrown
On Tue, May 02 2017, Ming Lei wrote: > On Tue, May 02, 2017 at 01:42:26PM +1000, NeilBrown wrote: >> blk_bio_segment_split() makes sure bios have no more than >> BIO_MAX_PAGES entries in the bi_io_vec. >> This was done because bio_clone_bioset() (when given a >> mempo

[PATCH 07/13 V2] drbd: use bio_clone_fast() instead of bio_clone()

2017-05-02 Thread NeilBrown
if the GFP flags allow for sleeping. Reviewed-by: Christoph Hellwig <h...@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- This patch needed to be refreshed after long lines were wrapped in an earlier patch. Also added the BIOSET_NEED_RESCUER flag for new bioset as it is

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-05-10 Thread NeilBrown
On Tue, May 02 2017, NeilBrown wrote: > This is a revision of my series of patches working > towards removing the bioset work queues. Hi Jens, could I get some feed-back about your thoughts on this series? Will you apply it? When? Do I need to resend anything? Would you like a gi

Re: [PATCH v4 13/27] lib: add errseq_t type and infrastructure for handling it

2017-05-09 Thread NeilBrown
ff Layton <jlay...@redhat.com> I like that this is a separate lib/*.c - nicely structured too. Reviewed-by: NeilBrown <ne...@suse.com> Thanks, NeilBrown > --- > include/linux/errseq.h | 19 + > lib/Makefile | 2 +- > lib/errseq.c | 199 > ++

Re: [PATCH 1/2] loop: use filp_close() rather than fput()

2017-06-17 Thread NeilBrown
On Sat, Jun 17 2017, Al Viro wrote: > On Fri, Jun 16, 2017 at 03:02:09PM +1000, NeilBrown wrote: >> When a loop device is being shutdown the backing file is >> closed with fput(). This is different from how close(2) >> closes files - it uses filp_close(). >> >

Re: [PATCH 0/2] Two fixes for loop devices

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/15/2017 11:02 PM, NeilBrown wrote: >> Hi Jens, >> one of these is a resend of a patch I sent a while back. >> The other is new - loop closes files differently from close() >> and in a way that can confuse NFS.

[PATCH 02/13] blk: replace bioset_create_nobvec() with a flags arg to bioset_create()

2017-06-17 Thread NeilBrown
adead.org> Reviewed-by: Ming Lei <ming@redhat.com> Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c | 60 +-- block/blk-core.c|2 + drivers/block/drbd/drbd_main.c |2 + drivers/

[PATCH 10/13] xen-blkfront: remove bio splitting.

2017-06-17 Thread NeilBrown
..@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/xen-blkfront.c | 54 ++ 1 file changed, 3 insertions(+), 51 deletions(-) diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index e3be666c2776..a

[PATCH 12/13] block: remove bio_clone() and all references.

2017-06-17 Thread NeilBrown
-by: Christoph Hellwig <h...@lst.de> Reviewed-by: Ming Lei <ming@redhat.com> Signed-off-by: NeilBrown <ne...@suse.com> --- Documentation/block/biodoc.txt |2 +- block/bio.c|2 +- block/blk-merge.c |6 +++--- drivers/md/md.c

[PATCH 09/13] lightnvm/pblk-read: use bio_clone_fast()

2017-06-17 Thread NeilBrown
Hellwig <h...@lst.de> Reviewed-by: Javier González <jav...@cnexlabs.com> Tested-by: Javier González <jav...@cnexlabs.com> Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/lightnvm/pblk-init.c | 12 +++- drivers/lightnvm/pblk-read.c |2 +- drivers/lightnvm

[PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
, and simple changes to compensate for a couple of changes to the block tree since the last posting. I hope to eventually get rid of the new BIOSET_NEED_RESCUER flag, but that needs work in dm and probably bcache first. Thanks, NeilBrown --- NeilBrown (13): blk: remove bio_set arg from

[PATCH 03/13] blk: make the bioset rescue_workqueue optional.

2017-06-17 Thread NeilBrown
<ming@redhat.com> (minor fixes) Reviewed-by: Ming Lei <ming@redhat.com> Signed-off-by: NeilBrown <ne...@suse.com> --- block/bio.c| 13 +++-- block/blk-core.c |3 ++- drivers/block/drbd/drbd_main.c |4 +++- drivers/md/bc

[PATCH 01/13] blk: remove bio_set arg from blk_queue_split()

2017-06-17 Thread NeilBrown
<jav...@cnexlabs.com> Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-core.c |2 +- block/blk-merge.c |9 - block/blk-mq.c|2 +- drivers/block/drbd/drbd_req.c |2 +- drivers/block/pktcdvd.c |2 +- drivers/block/

[PATCH 13/13] block: don't check for BIO_MAX_PAGES in blk_bio_segment_split()

2017-06-17 Thread NeilBrown
ed-off-by: NeilBrown <ne...@suse.com> --- block/blk-merge.c | 16 1 file changed, 16 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index e7862e9dcc39..cea544ec5d96 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -108,25 +108,9 @@ stati

[PATCH 08/13] pktcdvd: use bio_clone_fast() instead of bio_clone()

2017-06-17 Thread NeilBrown
, pkt_bio_set, can also be use for the bio_split() call as the two allocations (bio_clone_fast, and bio_split) are independent, neither can block a bio allocated by the other. Reviewed-by: Christoph Hellwig <h...@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/pktc

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/16/2017 01:34 AM, Christoph Hellwig wrote: >> On Fri, Jun 16, 2017 at 05:30:50PM +1000, NeilBrown wrote: >>> I've pushed the new version to the same place. Do you actually want >>> me to re-post all the patches? >>

[PATCH 06/13] rbd: use bio_clone_fast() instead of bio_clone()

2017-06-17 Thread NeilBrown
(). Reviewed-by: Christoph Hellwig <h...@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/block/rbd.c | 16 +++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 5420bc40c544..b008b6a98098 100644 --- a/d

Re: [PATCH 00/13] block: assorted cleanup for bio splitting and cloning.

2017-06-17 Thread NeilBrown
On Fri, Jun 16 2017, Jens Axboe wrote: > On 06/16/2017 01:34 AM, Christoph Hellwig wrote: >> On Fri, Jun 16, 2017 at 05:30:50PM +1000, NeilBrown wrote: >>> I've pushed the new version to the same place. Do you actually want >>> me to re-post all the patches? >>

[PATCH 11/13] bcache: use kmalloc to allocate bio in bch_data_verify()

2017-06-17 Thread NeilBrown
is used either for the bio or the pages. Reviewed-by: Christoph Hellwig <h...@lst.de> Acked-by: Kent Overstreet <kent.overstr...@gmail.com> Reviewed-by : Ming Lei <ming@redhat.com> Signed-off-by: NeilBrown <ne...@suse.com> --- drivers/md/bcache/debug.c |2 +- 1 f

[PATCH 05/13] block: Improvements to bounce-buffer handling

2017-06-17 Thread NeilBrown
ted using single bio_for_each_segment loop) Reviewed-by: Ming Lei <ming@redhat.com> Reviewed-by: Christoph Hellwig <h...@lst.de> Signed-off-by: NeilBrown <ne...@suse.com> --- block/blk-merge.c | 14 -- block/bounce.c| 32 ++-- 2 f

  1   2   >