RE: +AFs-PATCH+AF0- blk-mq: Set request mapping to NULL in blk+AF8-mq+AF8-put+AF8-driver+AF8-tag

2018-12-05 Thread Kashyap Desai
> > If the 'tag' passed to scsi_host_find_tag() is valid, I think there > shouldn't have such issue. > > If you want to find outstanding IOs, maybe you can try > blk_mq_queue_tag_busy_iter() > or blk_mq_tagset_busy_iter(), because you may not know if the passed 'tag' > to > scsi_host_find_tag() is

Re: DIF/DIX issue related to config CONFIG_SCSI_MQ_DEFAULT

2018-12-05 Thread Martin K. Petersen
John, >> I'll have to go revisit the archives for your exact fio bits. Your fio test case ran to completion on all my test systems. >> But I successfully ran our DIX/T10 PI qualification tooling without any >> errors on the latest SCSI tree. On both scsi_debug and FC hardware. > > I'm not fimi

[PATCH V10 4/4] blk-mq: replace and kill blk_mq_request_issue_directly

2018-12-05 Thread Jianchao Wang
Replace blk_mq_request_issue_directly with blk_mq_try_issue_directly in blk_insert_cloned_request and kill it as nobody uses it any more. Signed-off-by: Jianchao Wang --- block/blk-core.c | 4 +++- block/blk-mq.c | 9 + block/blk-mq.h | 6 -- 3 files changed, 8 insertions(+), 11

[PATCH V10 1/4] blk-mq: insert to hctx dispatch list when bypass_insert is true

2018-12-05 Thread Jianchao Wang
We don't allow direct dispatch of anything but regular reads/writes and insert all of non-read-write requests. However, this is not correct for 'bypass_insert == true' case where inserting is ignored and BLK_STS_RESOURCE is returned. The caller will fail forever. Fix it with inserting the non-read

[PATCH V10 0/4] blk-mq: refactor code of issue directly

2018-12-05 Thread Jianchao Wang
Hi Jens Please consider this patchset for 4.21. It refactors the code of issue request directly to unify the interface and make the code clearer and more readable. This patch set is rebased on the recent for-4.21/block and add the 1st patch which inserts the non-read-write request to hctx dispat

[PATCH V10 2/4] blk-mq: refactor the code of issue request directly

2018-12-05 Thread Jianchao Wang
Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly into one interface to unify the interfaces to issue requests directly. The merged interface takes over the requests totally, it could insert, end or do nothing based on the return value of .queue_rq and 'bypass' parameter. Then caller

[PATCH V10 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests

2018-12-05 Thread Jianchao Wang
It is not necessary to issue request directly with bypass 'true' in blk_mq_sched_insert_requests and handle the non-issued requests itself. Just set bypass to 'false' and let blk_mq_try_issue_directly handle them totally. Remove the blk_rq_can_direct_dispatch check, because blk_mq_try_issue_directl

Re: [PATCH 2/2] loop: Better discard support for block devices

2018-12-05 Thread Martin K. Petersen
Evan, > Ah, I see. But I think it's useful to reflect max_discard_sectors, > max_write_zeroes_sectors, discard_granularity, and discard_alignment > from the block device to the loop device. With the exception of > discard_alignment, these parameters are visible via sysfs, discard_alignment is v

Re: [PATCH v3 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-05 Thread Jens Axboe
On 12/5/18 1:24 PM, Mike Snitzer wrote: > Hi, > > This v3 adds "block: stop passing 'cpu' to all percpu stats methods" > before switching over to percpu counters. I gave this a shot, with the hope of being able to use it for blk-mq as well. The tldr is that it's still slower than the tag iteratio

Re: [PATCH V9 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests

2018-12-05 Thread Jens Axboe
On 12/5/18 6:11 PM, jianchao.wang wrote: >> And why aren't we just using the list_empty() check like before, and not >> having to add the inval cookie value? > > Because we use 'bypass == false' here, so blk_mq_try_issue_directly > will take over the request totally, so the request will always be

Re: [PATCH V9 2/4] blk-mq: refactor the code of issue request directly

2018-12-05 Thread jianchao.wang
On 12/6/18 12:27 AM, Jens Axboe wrote: > On 12/5/18 12:44 AM, Jianchao Wang wrote: >> Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly >> into one interface to unify the interfaces to issue requests >> directly. The merged interface takes over the requests totally, >> it could ins

Re: [PATCH V9 1/4] blk-mq: insert to hctx dispatch list when bypass_insert is true

2018-12-05 Thread jianchao.wang
On 12/6/18 12:25 AM, Jens Axboe wrote: > On 12/5/18 12:43 AM, Jianchao Wang wrote: >> We don't allow direct dispatch of anything but regular reads/writes >> and insert all of non-read-write requests. However, this is not >> correct for 'bypass_insert == true' case where inserting is ignored >> a

Re: [PATCH V9 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests

2018-12-05 Thread jianchao.wang
Hi Jens On 12/6/18 12:30 AM, Jens Axboe wrote: > On 12/5/18 12:44 AM, Jianchao Wang wrote: >> diff --git a/block/blk-mq.c b/block/blk-mq.c >> index fe92e52..0dfa269 100644 >> --- a/block/blk-mq.c >> +++ b/block/blk-mq.c >> @@ -1899,32 +1899,23 @@ blk_status_t blk_mq_request_issue_directly(struct

Re: +AFs-PATCH+AF0- blk-mq: Set request mapping to NULL in blk+AF8-mq+AF8-put+AF8-driver+AF8-tag

2018-12-05 Thread Ming Lei
On Tue, Dec 04, 2018 at 11:48:48PM +0530, Kashyap Desai wrote: > > -Original Message- > > From: Bart Van Assche [mailto:bvanass...@acm.org] > > Sent: Tuesday, December 4, 2018 10:45 PM > > To: Kashyap Desai; linux-block; Jens Axboe; Ming Lei; linux-scsi > > Cc: Suganath Prabu Subramani; Sre

Re: [PATCH 2/2] loop: Better discard support for block devices

2018-12-05 Thread Ming Lei
On Wed, Dec 05, 2018 at 11:35:57AM -0800, Evan Green wrote: > Hi Ming, > > On Tue, Dec 4, 2018 at 5:11 PM Ming Lei wrote: > > > > On Tue, Dec 04, 2018 at 02:19:46PM -0800, Evan Green wrote: > > > Hi Ming, > > > > > > On Tue, Nov 27, 2018 at 5:26 PM Ming Lei wrote: > > > > > > > > On Tue, Oct 30,

Re: [PATCH] block/025: test discard sector alignement and sector size overflow

2018-12-05 Thread Omar Sandoval
On Thu, Nov 15, 2018 at 12:00:17PM +0800, Ming Lei wrote: > This test covers the following two issues: > > 1) discard sector need to be aligned with logical block size > > 2) make sure 'sector_t' instead of 'unsigned int' is used when comparing > with discard sector size > > Signed-off-by: Ming

Re: [GIT PULL] Block fixes for 4.20-rc6

2018-12-05 Thread pr-tracker-bot
The pull request you sent on Wed, 5 Dec 2018 13:25:47 -0700: > git://git.kernel.dk/linux-block.git tags/for-linus-20181205 has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/4eaaa2b99e30305f4bc677f4abfe56c1f8b39670 Thank you! -- Deet-doot-dot, I am a bot. ht

[GIT PULL] Block fixes for 4.20-rc6

2018-12-05 Thread Jens Axboe
tty easily. Full explanation in the patch. None of the regular filesystem and storage testing has triggered it, even though it's been around since 4.19-rc1. Outside of that, whitelist trim tweak for certain Samsung devices for libata. Please pull! git://git.kernel.dk/linux-block.git tags/for

[PATCH v3 7/7] dm: remove the pending IO accounting

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka Remove the "pending" atomic counters, that duplicate block-core's in_flight counters, and update md_in_flight() to look at percpu in_flight counters. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm-core.h | 2 -- drivers/md/dm.c | 34 ++

[PATCH v3 4/7] block: delete part_round_stats and switch to less precise counting

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka We want to convert to per-cpu in_flight counters. The function part_round_stats needs the in_flight counter every jiffy, it would be too costly to sum all the percpu variables every jiffy, so it must be deleted. part_round_stats is used to calculate two counters - time_in_q

[PATCH v3 0/7] per-cpu in_flight counters for bio-based drivers

2018-12-05 Thread Mike Snitzer
Hi, This v3 adds "block: stop passing 'cpu' to all percpu stats methods" before switching over to percpu counters. Thanks, Mike Mike Snitzer (2): dm rq: leverage blk_mq_queue_busy() to check for outstanding IO block: stop passing 'cpu' to all percpu stats methods Mikulas Patocka (5): dm:

[PATCH v3 2/7] dm rq: leverage blk_mq_queue_busy() to check for outstanding IO

2018-12-05 Thread Mike Snitzer
Now that request-based dm-multipath only supports blk-mq, make use of the newly introduced blk_mq_queue_busy() to check for outstanding IO -- rather than (ab)using the block core's in_flight counters. Signed-off-by: Mike Snitzer --- drivers/md/dm-rq.c | 9 - 1 file changed, 4 insertions(

[PATCH v3 3/7] block: stop passing 'cpu' to all percpu stats methods

2018-12-05 Thread Mike Snitzer
All of part_stat_* and related methods are used with preempt disabled, so there is no need to pass cpu around to allow of them. Just call smp_processor_id() as needed. Suggested-by: Jens Axboe Signed-off-by: Mike Snitzer --- block/bio.c | 16 +--- block/blk-core.c

[PATCH v3 5/7] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka Now when part_round_stats is gone, we can switch to per-cpu in-flight counters. We use the local-atomic type local_t, so that if part_inc_in_flight or part_dec_in_flight is reentrantly called from an interrupt, the value will be correct. The other counters could be corrupt

[PATCH v3 1/7] dm: dont rewrite dm_disk(md)->part0.in_flight

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka generic_start_io_acct and generic_end_io_acct already update the variable in_flight using atomic operations, so we don't have to overwrite them again. Signed-off-by: Mikulas Patocka Signed-off-by: Mike Snitzer --- drivers/md/dm.c | 4 +--- 1 file changed, 1 insertion(+),

[PATCH v3 6/7] block: return just one value from part_in_flight

2018-12-05 Thread Mike Snitzer
From: Mikulas Patocka The previous patches deleted all the code that needed the second value returned from part_in_flight - now the kernel only uses the first value. Consequently, part_in_flight (and blk_mq_in_flight) may be changed so that it only returns one value. This patch just refactors t

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Jens Axboe
On 12/5/18 12:09 PM, Guenter Roeck wrote: > On Wed, Dec 05, 2018 at 10:59:21AM -0700, Jens Axboe wrote: > [ ... ] >> >>> Also, it seems to me that even with this problem fixed, blk-mq may not >>> be ready for primetime after all. With that in mind, maybe commit >>> d5038a13eca72 ("scsi: core: switc

WARNING: refcount bug in blk_mq_free_request

2018-12-05 Thread syzbot
Hello, syzbot found the following crash on: HEAD commit:60b548237fed Merge git://git.kernel.org/pub/scm/linux/kern.. git tree: upstream console output: https://syzkaller.appspot.com/x/log.txt?x=13dab87d40 kernel config: https://syzkaller.appspot.com/x/.config?x=c94f9f0c0363db4b da

Re: [PATCH 2/2] loop: Better discard support for block devices

2018-12-05 Thread Evan Green
Hi Ming, On Tue, Dec 4, 2018 at 5:11 PM Ming Lei wrote: > > On Tue, Dec 04, 2018 at 02:19:46PM -0800, Evan Green wrote: > > Hi Ming, > > > > On Tue, Nov 27, 2018 at 5:26 PM Ming Lei wrote: > > > > > > On Tue, Oct 30, 2018 at 04:06:24PM -0700, Evan Green wrote: > > > > If the backing device for a

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Guenter Roeck
On Wed, Dec 05, 2018 at 10:59:21AM -0700, Jens Axboe wrote: [ ... ] > > > Also, it seems to me that even with this problem fixed, blk-mq may not > > be ready for primetime after all. With that in mind, maybe commit > > d5038a13eca72 ("scsi: core: switch to scsi-mq by default") was a > > bit premat

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Jens Axboe
On 12/5/18 11:18 AM, Mike Snitzer wrote: > On Wed, Dec 05 2018 at 1:04pm -0500, > Jens Axboe wrote: > >> On 12/5/18 11:03 AM, Mike Snitzer wrote: >>> On Wed, Dec 05 2018 at 12:54pm -0500, >>> Jens Axboe wrote: >>> On 12/5/18 10:49 AM, Mike Snitzer wrote: > On Wed, Dec 05 2018 at 12:30p

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 1:04pm -0500, Jens Axboe wrote: > On 12/5/18 11:03 AM, Mike Snitzer wrote: > > On Wed, Dec 05 2018 at 12:54pm -0500, > > Jens Axboe wrote: > > > >> On 12/5/18 10:49 AM, Mike Snitzer wrote: > >>> On Wed, Dec 05 2018 at 12:30pm -0500, > >>> Jens Axboe wrote: > >>> >

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Jens Axboe
On 12/5/18 11:03 AM, Mike Snitzer wrote: > On Wed, Dec 05 2018 at 12:54pm -0500, > Jens Axboe wrote: > >> On 12/5/18 10:49 AM, Mike Snitzer wrote: >>> On Wed, Dec 05 2018 at 12:30pm -0500, >>> Jens Axboe wrote: >>> There's also no need to pass in the cpu, if we're not running with pree

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 12:54pm -0500, Jens Axboe wrote: > On 12/5/18 10:49 AM, Mike Snitzer wrote: > > On Wed, Dec 05 2018 at 12:30pm -0500, > > Jens Axboe wrote: > > > >> There's also no need to pass in the cpu, if we're not running with > >> preempt disabled already we have a problem. > > >

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Jens Axboe
On 12/5/18 10:55 AM, Guenter Roeck wrote: > On Tue, Dec 04, 2018 at 07:25:05PM -0700, Jens Axboe wrote: >> On 12/4/18 6:38 PM, Guenter Roeck wrote: >>> On Tue, Dec 04, 2018 at 03:47:46PM -0700, Jens Axboe wrote: If we attempt a direct issue to a SCSI device, and it returns BUSY, then we q

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Guenter Roeck
On Tue, Dec 04, 2018 at 07:25:05PM -0700, Jens Axboe wrote: > On 12/4/18 6:38 PM, Guenter Roeck wrote: > > On Tue, Dec 04, 2018 at 03:47:46PM -0700, Jens Axboe wrote: > >> If we attempt a direct issue to a SCSI device, and it returns BUSY, then > >> we queue the request up normally. However, the SC

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Jens Axboe
On 12/5/18 10:49 AM, Mike Snitzer wrote: > On Wed, Dec 05 2018 at 12:30pm -0500, > Jens Axboe wrote: > >> On 11/30/18 3:22 PM, Mike Snitzer wrote: >>> diff --git a/block/genhd.c b/block/genhd.c >>> index cdf174d7d329..d4c9dd65def6 100644 >>> --- a/block/genhd.c >>> +++ b/block/genhd.c >>> @@ -45,

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Mike Snitzer
On Wed, Dec 05 2018 at 12:30pm -0500, Jens Axboe wrote: > On 11/30/18 3:22 PM, Mike Snitzer wrote: > > diff --git a/block/genhd.c b/block/genhd.c > > index cdf174d7d329..d4c9dd65def6 100644 > > --- a/block/genhd.c > > +++ b/block/genhd.c > > @@ -45,53 +45,76 @@ static void disk_add_events(struct

Re: [PATCH 0/3] Unify the throttling code for wbt and io-latency

2018-12-05 Thread Jens Axboe
On 12/4/18 10:59 AM, Josef Bacik wrote: > Originally when I wrote io-latency and the rq_qos code to provide a common > base > between wbt and io-latency I left out the throttling part. These were > basically > the same, but slightly different in both cases. The difference was enough and > the c

Re: [PATCH v2 4/6] block: switch to per-cpu in-flight counters

2018-12-05 Thread Jens Axboe
On 11/30/18 3:22 PM, Mike Snitzer wrote: > diff --git a/block/genhd.c b/block/genhd.c > index cdf174d7d329..d4c9dd65def6 100644 > --- a/block/genhd.c > +++ b/block/genhd.c > @@ -45,53 +45,76 @@ static void disk_add_events(struct gendisk *disk); > static void disk_del_events(struct gendisk *disk);

Re: [PATCH v6 00/14] block: always associate blkg and refcount cleanup

2018-12-05 Thread Jens Axboe
On 12/5/18 10:10 AM, Dennis Zhou wrote: > Hi everyone, > > This is a minor update addressing the feedback from Mike and Josef. > > v6: > 0005: Address Mike's feedback for the flush bio comment. > 0006: Add a WARN_ONCE() for blkg fallback suggested by Josef. Thanks Dennis, applied for 4.21. --

[PATCH 03/14] blkcg: convert blkg_lookup_create() to find closest blkg

2018-12-05 Thread Dennis Zhou
There are several scenarios where blkg_lookup_create() can fail such as the blkcg dying, request_queue is dying, or simply being OOM. Most handle this by simply falling back to the q->root_blkg and calling it a day. This patch implements the notion of closest blkg. During blkg_lookup_create(), if

[PATCH 01/14] blkcg: fix ref count issue with bio_blkcg() using task_css

2018-12-05 Thread Dennis Zhou
The bio_blkcg() function turns out to be inconsistent and consequently dangerous to use. The first part returns a blkcg where a reference is owned by the bio meaning it does not need to be rcu protected. However, the third case, the last line, is problematic: return css_to_blkcg(task_css(c

[PATCH 06/14] blkcg: associate blkg when associating a device

2018-12-05 Thread Dennis Zhou
Previously, blkg association was handled by controller specific code in blk-throttle and blk-iolatency. However, because a blkg represents a relationship between a blkcg and a request_queue, it makes sense to keep the blkg->q and bio->bi_disk->queue consistent. This patch moves association into th

[PATCH 09/14] blkcg: associate writeback bios with a blkg

2018-12-05 Thread Dennis Zhou
One of the goals of this series is to remove a separate reference to the css of the bio. This can and should be accessed via bio_blkcg(). In this patch, wbc_init_bio() now requires a bio to have a device associated with it. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo

[PATCH 05/14] dm: set the static flush bio device on demand

2018-12-05 Thread Dennis Zhou
The next patch changes the macro bio_set_dev() to associate a bio with a blkg based on the device set. However, dm creates a static bio to be used as the basis for cloning empty flush bios on creation. The bio_set_dev() call in alloc_dev() will cause problems with the next patch adding association

[PATCH 02/14] blkcg: update blkg_lookup_create() to do locking

2018-12-05 Thread Dennis Zhou
To know when to create a blkg, the general pattern is to do a blkg_lookup() and if that fails, lock and do the lookup again, and if that fails finally create. It doesn't make much sense for everyone who wants to do creation to write this themselves. This changes blkg_lookup_create() to do locking

[PATCH 08/14] blkcg: associate a blkg for pages being evicted by swap

2018-12-05 Thread Dennis Zhou
A prior patch in this series added blkg association to bios issued by cgroups. There are two other paths that we want to attribute work back to the appropriate cgroup: swap and writeback. Here we modify the way swap tags bios to include the blkg. Writeback will be tackle in the next patch. Signed-

[PATCH 07/14] blkcg: consolidate bio_issue_init() to be a part of core

2018-12-05 Thread Dennis Zhou
bio_issue_init among other things initializes the timestamp for an IO. Rather than have this logic handled by policies, this consolidates it to be on the init paths (normal, clone, bounce clone). Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Liu Bo Reviewed-by: Josef Bacik --- b

[PATCH 14/14] blkcg: rename blkg_try_get() to blkg_tryget()

2018-12-05 Thread Dennis Zhou
blkg reference counting now uses percpu_ref rather than atomic_t. Let's make this consistent with css_tryget. This renames blkg_try_get to blkg_tryget and now returns a bool rather than the blkg or %NULL. Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik Acked-by: Tejun Heo --- block/bio.c

[PATCH 11/14] blkcg: remove additional reference to the css

2018-12-05 Thread Dennis Zhou
The previous patch in this series removed carrying around a pointer to the css in blkg. However, the blkg association logic still relied on taking a reference on the css to ensure we wouldn't fail in getting a reference for the blkg. Here the implicit dependency on the css is removed. The associat

[PATCH v6 00/14] block: always associate blkg and refcount cleanup

2018-12-05 Thread Dennis Zhou
Hi everyone, This is a minor update addressing the feedback from Mike and Josef. v6: 0005: Address Mike's feedback for the flush bio comment. 0006: Add a WARN_ONCE() for blkg fallback suggested by Josef. >From v5: A special case with dm is the flush bio which is statically initialized before the

[PATCH 04/14] blkcg: introduce common blkg association logic

2018-12-05 Thread Dennis Zhou
There are 3 ways blkg association can happen: association with the current css, with the page css (swap), or from the wbc css (writeback). This patch handles how association is done for the first case where we are associating bsaed on the current css. If there is already a blkg associated, the css

[PATCH 10/14] blkcg: remove bio->bi_css and instead use bio->bi_blkg

2018-12-05 Thread Dennis Zhou
Prior patches ensured that any bio that interacts with a request_queue is properly associated with a blkg. This makes bio->bi_css unnecessary as blkg maintains a reference to blkcg already. This removes the bio field bi_css and transfers corresponding uses to access via bi_blkg. Signed-off-by: De

[PATCH 12/14] blkcg: remove bio_disassociate_task()

2018-12-05 Thread Dennis Zhou
Now that a bio only holds a blkg reference, so clean up is simply putting back that reference. Remove bio_disassociate_task() as it just calls bio_disassociate_blkg() and call the latter directly. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/bio.c |

[PATCH 13/14] blkcg: change blkg reference counting to use percpu_ref

2018-12-05 Thread Dennis Zhou
Every bio is now associated with a blkg putting blkg_get, blkg_try_get, and blkg_put on the hot path. Switch over the refcnt in blkg to use percpu_ref. Signed-off-by: Dennis Zhou Acked-by: Tejun Heo Reviewed-by: Josef Bacik --- block/blk-cgroup.c | 41 ++

Re: [PATCH V9 4/4] blk-mq: replace and kill blk_mq_request_issue_directly

2018-12-05 Thread Jens Axboe
On 12/5/18 12:44 AM, Jianchao Wang wrote: > Replace blk_mq_request_issue_directly with blk_mq_try_issue_directly > in blk_insert_cloned_request and kill it as nobody uses it any more. Looks good. -- Jens Axboe

Re: [PATCH V9 3/4] blk-mq: issue directly with bypass 'false' in blk_mq_sched_insert_requests

2018-12-05 Thread Jens Axboe
On 12/5/18 12:44 AM, Jianchao Wang wrote: > diff --git a/block/blk-mq.c b/block/blk-mq.c > index fe92e52..0dfa269 100644 > --- a/block/blk-mq.c > +++ b/block/blk-mq.c > @@ -1899,32 +1899,23 @@ blk_status_t blk_mq_request_issue_directly(struct > request *rq, bool last) > void blk_mq_try_issue_list

Re: [PATCH V9 2/4] blk-mq: refactor the code of issue request directly

2018-12-05 Thread Jens Axboe
On 12/5/18 12:44 AM, Jianchao Wang wrote: > Merge blk_mq_try_issue_directly and __blk_mq_try_issue_directly > into one interface to unify the interfaces to issue requests > directly. The merged interface takes over the requests totally, > it could insert, end or do nothing based on the return value

Re: [PATCH V9 1/4] blk-mq: insert to hctx dispatch list when bypass_insert is true

2018-12-05 Thread Jens Axboe
On 12/5/18 12:43 AM, Jianchao Wang wrote: > We don't allow direct dispatch of anything but regular reads/writes > and insert all of non-read-write requests. However, this is not > correct for 'bypass_insert == true' case where inserting is ignored > and BLK_STS_RESOURCE is returned. The caller will

Re: [PATCH 1/2] block: get rid of BLK_MAX_TIMEOUT

2018-12-05 Thread Bart Van Assche
On Wed, 2018-12-05 at 23:37 +0800, Weiping Zhang wrote: > @@ -130,7 +119,7 @@ void blk_add_timer(struct request *req) >* than an existing one, modify the timer. Round up to next nearest >* second. >*/ > - expiry = blk_rq_timeout(round_jiffies_up(expiry)); > + expiry

[PATCH 1/2] block: get rid of BLK_MAX_TIMEOUT

2018-12-05 Thread Weiping Zhang
Since io_timeout was added to sysfs, the user can tune timeouts by that attribute, so kill this limitation. Signed-off-by: Weiping Zhang --- block/blk-timeout.c | 13 + block/blk.h | 4 2 files changed, 1 insertion(+), 16 deletions(-) diff --git a/block/blk-timeout.c b

[PATCH 2/2] block: add BLK_DEF_TIMEOUT

2018-12-05 Thread Weiping Zhang
Add an obvious definition for default request timeout. Signed-off-by: Weiping Zhang --- block/blk-mq.c | 2 +- block/blk.h| 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index 900550594651..53337d5c37af 100644 --- a/block/blk-mq.c +++ b/

[PATCH 0/2] get rid of BLK_MAX_TIMEOUT

2018-12-05 Thread Weiping Zhang
Get rid of BLK_MAX_TIMEOUT, since we want use io_timeout to control the maximun timeouts of a request, so remove this limitation. Weiping Zhang (2): block: get rid of BLK_MAX_TIMEOUT block: add BLK_DEF_TIMEOUT block/blk-mq.c | 2 +- block/blk-timeout.c | 13 + block/blk.h

[PATCH 2/2] blktests: block/025: an io.latency test

2018-12-05 Thread Josef Bacik
This is a test to verify io.latency is working properly. It does this by first running a fio job by itself to figure out how fast it runs. Then we calculate some thresholds, set up 2 cgroups, a fast and a slow cgroup, and then run the same job in both groups simultaneously. We should see the slow

[PATCH 1/2] blktests: add cgroup2 infrastructure

2018-12-05 Thread Josef Bacik
In order to test io.latency and other cgroup related things we need some supporting helpers to setup and tear down cgroup2. This adds support for checking that we can even configure cgroup2 things, set them up if need be, and then add the cleanup stuff to the main cleanup function so everything is

[PATCH 0/2][V2] io.latency test for blktests

2018-12-05 Thread Josef Bacik
v1->v2: - dropped my python library, TIL about jq. - fixed the spelling mistakes in the test. -- Original message -- This patchset is to add a test to verify io.latency is working properly, and to add all the supporting code to run that test. First is the cgroup2 infrastructure which is fairly s

Re: [PATCH] Fix sync. in inode_has_no_xattr()

2018-12-05 Thread Jan Kara
On Wed 05-12-18 12:45:06, Alexander Lochmann wrote: > > inode.i_flags might be altered without proper > synchronisation when the inode belongs to devtmpfs. > The following stacktrace shows how to get there: > 13: entry_SYSENTER_32:460 > 12: do_fast_syscall_32:410 > 11: _static_cpu_has:146 > 10: do

Re: DIF/DIX issue related to config CONFIG_SCSI_MQ_DEFAULT

2018-12-05 Thread John Garry
On 05/12/2018 02:22, Martin K. Petersen wrote: Hi Martin, John, OK, great. Happy to help. So far we have hisi_sas fio issue to go on plus knowledge that this issue seems to be exposed/triggered by enabling SCSI MQ. I'll have to go revisit the archives for your exact fio bits. But I succ

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Jens Axboe
On 12/5/18 7:41 AM, Christoph Hellwig wrote: > On Tue, Dec 04, 2018 at 03:47:46PM -0700, Jens Axboe wrote: >> If we attempt a direct issue to a SCSI device, and it returns BUSY, then >> we queue the request up normally. However, the SCSI layer may have >> already setup SG tables etc for this partic

Re: [PATCH v3] block: add documentation for io_timeout

2018-12-05 Thread Weiping Zhang
Weiping Zhang 于2018年12月5日周三 下午10:49写道: > > Christoph Hellwig 于2018年12月5日周三 下午10:40写道: > > > > Can you please also send a patch to not show this attribute for > > drivers without a timeout handler? Thanks! > > Is there a simple way do that ? Shall we return -ENOTSUPP when user read/write this at

Re: [PATCH v3] block: add documentation for io_timeout

2018-12-05 Thread Weiping Zhang
Christoph Hellwig 于2018年12月5日周三 下午10:40写道: > > Can you please also send a patch to not show this attribute for > drivers without a timeout handler? Thanks! > Do you means checking q->mq_ops->timeout is NULL or not ? static void blk_mq_rq_timed_out(struct request *req, bool reserved) { re

Re: [PATCH] blk-mq: fix corruption with direct issue

2018-12-05 Thread Christoph Hellwig
On Tue, Dec 04, 2018 at 03:47:46PM -0700, Jens Axboe wrote: > If we attempt a direct issue to a SCSI device, and it returns BUSY, then > we queue the request up normally. However, the SCSI layer may have > already setup SG tables etc for this particular command. If we later > merge with this reques

Re: [PATCH v3] block: add documentation for io_timeout

2018-12-05 Thread Christoph Hellwig
Can you please also send a patch to not show this attribute for drivers without a timeout handler? Thanks! On Wed, Dec 05, 2018 at 10:17:06PM +0800, Weiping Zhang wrote: > Add documentation for /sys/block//queue/io_timeout. > > Signed-off-by: Weiping Zhang > --- > Documentation/ABI/testing/sys

[PATCH v3] block: add documentation for io_timeout

2018-12-05 Thread Weiping Zhang
Add documentation for /sys/block//queue/io_timeout. Signed-off-by: Weiping Zhang --- Documentation/ABI/testing/sysfs-block | 10 ++ Documentation/block/queue-sysfs.txt | 7 +++ 2 files changed, 17 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-block b/Documentation/A

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Kirill Tkhai
; >>>>> Traceback is below, config and reproducer (not minimal, just a random one >>>>> populating swap) are attached. >>>>> >>>>> [ 29.097612] BUG: unable to handle kernel NULL pointer dereference at >>>>> &g

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Kirill Tkhai
>>>> populating swap) are attached. >>>> >>>> [ 29.097612] BUG: unable to handle kernel NULL pointer dereference at >>>> >>>> [ 29.098730] #PF error: [INSTR] >>>> [ 29.099104] PGD 0 P4D 0 >>>>

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Jens Axboe
On 12/5/18 6:47 AM, Christoph Hellwig wrote: > On Wed, Dec 05, 2018 at 06:39:26AM -0700, Jens Axboe wrote: >>> What device is this? >> >> This might also help... > > Yes, it should. I had missed that we turned on QUEUE_FLAG_POLL > by default, which is rather odd. The even weirder things is that

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Christoph Hellwig
On Wed, Dec 05, 2018 at 06:39:26AM -0700, Jens Axboe wrote: > > What device is this? > > This might also help... Yes, it should. I had missed that we turned on QUEUE_FLAG_POLL by default, which is rather odd. The even weirder things is that git-blame claims it was me who enabled it :) > > dif

Re: [PATCH v9] virtio_blk: add discard and write zeroes support

2018-12-05 Thread Michael S. Tsirkin
On Wed, Dec 05, 2018 at 09:46:16AM +, Liu, Changpeng wrote: > What's the status of this patch ? anybody pulled it for the branch ? I will merge it for next. > > > -Original Message- > > From: Stefan Hajnoczi [mailto:stefa...@redhat.com] > > Sent: Friday, November 2, 2018 12:18 PM >

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Jens Axboe
>>>> populating swap) are attached. >>>> >>>> [ 29.097612] BUG: unable to handle kernel NULL pointer dereference at >>>> >>>> [ 29.098730] #PF error: [INSTR] >>>> [ 29.099104] PGD 0 P4D 0 >>>>

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Jens Axboe
e to handle kernel NULL pointer dereference at >>> >>> [ 29.098730] #PF error: [INSTR] >>> [ 29.099104] PGD 0 P4D 0 >>> [ 29.099425] Oops: 0010 [#1] PREEMPT SMP >>> [ 29.099879] CPU: 3 PID: 925 Comm: bash Not tainted >>&

Re: [PATCH 15/26] aio: support for IO polling

2018-12-05 Thread Jens Axboe
On 12/5/18 2:56 AM, Benny Halevy wrote: >> +static int aio_iopoll_getevents(struct kioctx *ctx, >> +struct io_event __user *event, >> +unsigned int *nr_events, long min, long max) >> +{ >> +struct aio_kiocb *iocb; >> +int to_poll, poll

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Kirill Tkhai
#PF error: [INSTR] >> [ 29.099104] PGD 0 P4D 0 >> [ 29.099425] Oops: 0010 [#1] PREEMPT SMP >> [ 29.099879] CPU: 3 PID: 925 Comm: bash Not tainted >> 4.20.0-rc5-next-20181205+ #244 >> [ 29.100658] RIP: 0010: (null) >> [ 29.101100] Code: Bad RIP va

Re: Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Jens Axboe
gt; [ 29.099879] CPU: 3 PID: 925 Comm: bash Not tainted > 4.20.0-rc5-next-20181205+ #244 > [ 29.100658] RIP: 0010: (null) > [ 29.101100] Code: Bad RIP value. > [ 29.101480] RSP: :c923fb80 EFLAGS: 00010202 > [ 29.102061] RAX: 8182

Kernel crashes after 529262d56dbe "block: remove ->poll_fn"

2018-12-05 Thread Kirill Tkhai
BUG: unable to handle kernel NULL pointer dereference at [ 29.098730] #PF error: [INSTR] [ 29.099104] PGD 0 P4D 0 [ 29.099425] Oops: 0010 [#1] PREEMPT SMP [ 29.099879] CPU: 3 PID: 925 Comm: bash Not tainted 4.20.0-rc5-next-20181205+ #244 [ 29.100658] RIP: 0010: (n

Re: [PATCH V2] mm: Replace all open encodings for NUMA_NO_NODE

2018-12-05 Thread Lubomir Rintel
On Wed, 2018-12-05 at 17:01 +0530, Anshuman Khandual wrote: > > On 12/05/2018 02:56 AM, Lubomir Rintel wrote: > > On Mon, 2018-11-26 at 17:56 +0530, Anshuman Khandual wrote: > > > At present there are multiple places where invalid node number is encoded > > > as -1. Even though implicitly understo

Re: [PATCH V2] mm: Replace all open encodings for NUMA_NO_NODE

2018-12-05 Thread Anshuman Khandual
On 12/05/2018 02:56 AM, Lubomir Rintel wrote: > On Mon, 2018-11-26 at 17:56 +0530, Anshuman Khandual wrote: >> At present there are multiple places where invalid node number is encoded >> as -1. Even though implicitly understood it is always better to have macros >> in there. Replace these open

Re: [LKP] [mm] 19717e78a0: stderr.if(target_node==NUMA_NO_NODE){

2018-12-05 Thread Anshuman Khandual
On 12/05/2018 10:30 AM, kernel test robot wrote: > FYI, we noticed the following commit (built with gcc-7): > > commit: 19717e78a04d51512cf0e7b9b09c61f06b2af071 ("[PATCH V2] mm: Replace all > open encodings for NUMA_NO_NODE") > url: > https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm

Re: [PATCH 15/26] aio: support for IO polling

2018-12-05 Thread Benny Halevy
On Tue, 2018-12-04 at 16:37 -0700, Jens Axboe wrote: > Add polled variants of PREAD/PREADV and PWRITE/PWRITEV. These act > like their non-polled counterparts, except we expect to poll for > completion of them. The polling happens at io_getevent() time, and > works just like non-polled IO. > > To s

RE: [PATCH v9] virtio_blk: add discard and write zeroes support

2018-12-05 Thread Liu, Changpeng
What's the status of this patch ? anybody pulled it for the branch ? > -Original Message- > From: Stefan Hajnoczi [mailto:stefa...@redhat.com] > Sent: Friday, November 2, 2018 12:18 PM > To: Daniel Verkamp > Cc: virtualizat...@lists.linux-foundation.org; linux-block@vger.kernel.org; > Mic

Re: [PATCH 2/3] blktests: add python scripts for parsing fio json output

2018-12-05 Thread Johannes Thumshirn
On 04/12/2018 18:47, Josef Bacik wrote: > +For example > +"write" : { > +"io_bytes" : 313360384, > +"bw" : 1016, > +} > + > +Get's collapsed to > + > +"write_io_bytes" : 313360384, > +"write_bw": 1016, I'd definitively prefer to not p