Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-12 Thread Ming Lei
On Mon, May 11, 2020 at 11:26:07PM -0700, Sagi Grimberg wrote: > > > > devices will benefit from the batching so maybe the flag needs to be > > > inverted? BLK_MQ_F_DONT_BATCHING_SUBMISSION? > > > > Actually I'd rather to not add any flag, and we may use some algorithm > > (maybe EWMA or other

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-12 Thread Sagi Grimberg
devices will benefit from the batching so maybe the flag needs to be inverted? BLK_MQ_F_DONT_BATCHING_SUBMISSION? Actually I'd rather to not add any flag, and we may use some algorithm (maybe EWMA or other intelligent stuff, or use hctx->dispatch_busy directly) to figure out one dynamic

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-11 Thread Ming Lei
On Mon, May 11, 2020 at 02:23:14AM -0700, Sagi Grimberg wrote: > > > > > Basically, my idea is to dequeue request one by one, and for each > > > > dequeued request: > > > > > > > > - we try to get a budget and driver tag, if both succeed, add the > > > > request to one per-task list which can be

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-11 Thread Sagi Grimberg
Basically, my idea is to dequeue request one by one, and for each dequeued request: - we try to get a budget and driver tag, if both succeed, add the request to one per-task list which can be stored in stack variable, then continue to dequeue more request - if either budget or driver tag

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-10 Thread Ming Lei
On Sun, May 10, 2020 at 12:44:53AM -0700, Sagi Grimberg wrote: > > > > > > You're mostly correct. This is exactly why an I/O scheduler may be > > > > > applicable here IMO. Mostly because I/O schedulers tend to optimize > > > > > for > > > > > something specific and always present tradeoffs.

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-10 Thread Sagi Grimberg
You're mostly correct. This is exactly why an I/O scheduler may be applicable here IMO. Mostly because I/O schedulers tend to optimize for something specific and always present tradeoffs. Users need to understand what they are optimizing for. Hence I'd say this functionality can definitely be

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-09 Thread Ming Lei
On Sat, May 09, 2020 at 04:57:48PM +0800, Baolin Wang wrote: > On Sat, May 9, 2020 at 7:22 AM Ming Lei wrote: > > > > Hi Sagi, > > > > On Fri, May 08, 2020 at 03:19:45PM -0700, Sagi Grimberg wrote: > > > Hey Ming, > > > > > > > > Would it make sense to elevate this flag to a request_queue flag >

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-09 Thread Baolin Wang
On Sat, May 9, 2020 at 7:22 AM Ming Lei wrote: > > Hi Sagi, > > On Fri, May 08, 2020 at 03:19:45PM -0700, Sagi Grimberg wrote: > > Hey Ming, > > > > > > Would it make sense to elevate this flag to a request_queue flag > > > > (QUEUE_FLAG_ALWAYS_COMMIT)? > > > > > > request queue flag usually is

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-08 Thread Ming Lei
Hi Sagi, On Fri, May 08, 2020 at 03:19:45PM -0700, Sagi Grimberg wrote: > Hey Ming, > > > > Would it make sense to elevate this flag to a request_queue flag > > > (QUEUE_FLAG_ALWAYS_COMMIT)? > > > > request queue flag usually is writable, however this case just needs > > one read-only flag, so

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-08 Thread Sagi Grimberg
Hey Ming, Would it make sense to elevate this flag to a request_queue flag (QUEUE_FLAG_ALWAYS_COMMIT)? request queue flag usually is writable, however this case just needs one read-only flag, so I think it may be better to make it as tagset/hctx flag. I actually intended it to be writable.

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-08 Thread Ming Lei
On Fri, May 08, 2020 at 02:35:35PM -0700, Sagi Grimberg wrote: > > > > diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h > > > index f389d7c724bd..6a20f8e8eb85 100644 > > > --- a/include/linux/blk-mq.h > > > +++ b/include/linux/blk-mq.h > > > @@ -391,6 +391,7 @@ struct blk_mq_ops { > >

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-05-08 Thread Sagi Grimberg
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index f389d7c724bd..6a20f8e8eb85 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -391,6 +391,7 @@ struct blk_mq_ops { enum { BLK_MQ_F_SHOULD_MERGE = 1 << 0, BLK_MQ_F_TAG_SHARED = 1 << 1, +

Re: [RFC PATCH v2 1/7] block: Extand commit_rqs() to do batch processing

2020-04-28 Thread Baolin Wang
On Mon, Apr 27, 2020 at 11:46 PM Christoph Hellwig wrote: > > extand in the subject really shpuld be 'extend' Sorry for typo, and will fix in next version. > > On Sun, Apr 26, 2020 at 05:38:54PM +0800, Baolin Wang wrote: > > From: Ming Lei > > > > Now some SD/MMC host controllers can support