Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Ming Lei 
On Wed, Nov 22, 2017 at 04:47:48PM +, Bart Van Assche wrote:
> On Wed, 2017-11-22 at 13:11 +0800, Ming Lei wrote:
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index 11097477eeab..3d3797327491 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -161,6 +161,8 @@ void blk_freeze_queue(struct request_queue *q)
> >  * exported to drivers as the only user for unfreeze is blk_mq.
> >  */
> > blk_freeze_queue_start(q);
> > +   if (!q->mq_ops)
> > +   blk_drain_queue(q);
> > blk_mq_freeze_queue_wait(q);
> >  }
> 
> Since q_usage_counter now tracks legacy requests, is there any reason why we
> still need __blk_drain_queue()? Have you considered to eliminate
> __blk_drain_queue() and to call blk_run_queue() from inside blk_freeze_queue()
> instead of calling blk_drain_queue()? I'm asking this because

Yeah, that looks better, I am thinking of that too, will do this way
in V2.

-- 
Ming

Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Bart Van Assche 
On Wed, 2017-11-22 at 13:11 +0800, Ming Lei wrote:
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 11097477eeab..3d3797327491 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -161,6 +161,8 @@ void blk_freeze_queue(struct request_queue *q)
>* exported to drivers as the only user for unfreeze is blk_mq.
>*/
>   blk_freeze_queue_start(q);
> + if (!q->mq_ops)
> + blk_drain_queue(q);
>   blk_mq_freeze_queue_wait(q);
>  }

Since q_usage_counter now tracks legacy requests, is there any reason why we
still need __blk_drain_queue()? Have you considered to eliminate
__blk_drain_queue() and to call blk_run_queue() from inside blk_freeze_queue()
instead of calling blk_drain_queue()? I'm asking this because
blk_mq_freeze_queue_wait() uses a more efficient mechanism (wait_event())
than __blk_drain_queue() (while (...) msleep(10)).

Bart.

Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-22 Thread Ming Lei 
On Wed, Nov 22, 2017 at 08:04:13AM +0100, Hannes Reinecke wrote:
> On 11/22/2017 06:11 AM, Ming Lei wrote:
> > Now we track legacy requests with .q_usage_counter in commit 055f6e18e08f
> > ("block: Make q_usage_counter also track legacy requests"), but that
> > commit never runs and drains legacy queue before waiting for this counter
> > becoming zero, then IO hang is caused in the test of pulling disk during IO.
> > 
> > This patch fixes the issue by draining requests before waiting for
> > q_usage_counter becoming zero.
> > 
> > Fixes: 055f6e18e08f("block: Make q_usage_counter also track legacy 
> > requests")
> > Cc: Wen Xiong 
> > Signed-off-by: Ming Lei 
> > ---
> >  block/blk-core.c | 9 +++--
> >  block/blk-mq.c   | 2 ++
> >  block/blk.h  | 2 ++
> >  3 files changed, 11 insertions(+), 2 deletions(-)
> > 
> > diff --git a/block/blk-core.c b/block/blk-core.c
> > index 1038706edd87..5ae5ea0dca4c 100644
> > --- a/block/blk-core.c
> > +++ b/block/blk-core.c
> > @@ -562,6 +562,13 @@ static void __blk_drain_queue(struct request_queue *q, 
> > bool drain_all)
> > }
> >  }
> >  
> > +void blk_drain_queue(struct request_queue *q)
> > +{
> > +   spin_lock_irq(q->queue_lock);
> > +   __blk_drain_queue(q, true);
> > +   spin_unlock_irq(q->queue_lock);
> > +}
> > +
> >  /**
> >   * blk_queue_bypass_start - enter queue bypass mode
> >   * @q: queue of interest
> > @@ -689,8 +696,6 @@ void blk_cleanup_queue(struct request_queue *q)
> >  */
> > blk_freeze_queue(q);
> > spin_lock_irq(lock);
> > -   if (!q->mq_ops)
> > -   __blk_drain_queue(q, true);
> > queue_flag_set(QUEUE_FLAG_DEAD, q);
> > spin_unlock_irq(lock);
> >  
> > diff --git a/block/blk-mq.c b/block/blk-mq.c
> > index 11097477eeab..3d3797327491 100644
> > --- a/block/blk-mq.c
> > +++ b/block/blk-mq.c
> > @@ -161,6 +161,8 @@ void blk_freeze_queue(struct request_queue *q)
> >  * exported to drivers as the only user for unfreeze is blk_mq.
> >  */
> > blk_freeze_queue_start(q);
> > +   if (!q->mq_ops)
> > +   blk_drain_queue(q);
> > blk_mq_freeze_queue_wait(q);
> >  }
> >  
> > diff --git a/block/blk.h b/block/blk.h
> > index 3f1446937aec..442098aa9463 100644
> > --- a/block/blk.h
> > +++ b/block/blk.h
> > @@ -330,4 +330,6 @@ static inline void blk_queue_bounce(struct 
> > request_queue *q, struct bio **bio)
> >  }
> >  #endif /* CONFIG_BOUNCE */
> >  
> > +extern void blk_drain_queue(struct request_queue *q);
> > +
> >  #endif /* BLK_INTERNAL_H */
> > 
> I seem to have missed something.
> Is blk_drain_queue() ever used?

It is always called in blk_cleanup_queue().

Before 055f6e18e08f("block: Make q_usage_counter also track legacy requests",
the q_usage_counter just works for sync IO(such as dax) because we only
hold the refcount when calling q->make_request_fn(), so blk_drain_queue()
does do the job always.

Thanks,
Ming

Re: [PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-21 Thread Hannes Reinecke 
On 11/22/2017 06:11 AM, Ming Lei wrote:
> Now we track legacy requests with .q_usage_counter in commit 055f6e18e08f
> ("block: Make q_usage_counter also track legacy requests"), but that
> commit never runs and drains legacy queue before waiting for this counter
> becoming zero, then IO hang is caused in the test of pulling disk during IO.
> 
> This patch fixes the issue by draining requests before waiting for
> q_usage_counter becoming zero.
> 
> Fixes: 055f6e18e08f("block: Make q_usage_counter also track legacy requests")
> Cc: Wen Xiong 
> Signed-off-by: Ming Lei 
> ---
>  block/blk-core.c | 9 +++--
>  block/blk-mq.c   | 2 ++
>  block/blk.h  | 2 ++
>  3 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 1038706edd87..5ae5ea0dca4c 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -562,6 +562,13 @@ static void __blk_drain_queue(struct request_queue *q, 
> bool drain_all)
>   }
>  }
>  
> +void blk_drain_queue(struct request_queue *q)
> +{
> + spin_lock_irq(q->queue_lock);
> + __blk_drain_queue(q, true);
> + spin_unlock_irq(q->queue_lock);
> +}
> +
>  /**
>   * blk_queue_bypass_start - enter queue bypass mode
>   * @q: queue of interest
> @@ -689,8 +696,6 @@ void blk_cleanup_queue(struct request_queue *q)
>*/
>   blk_freeze_queue(q);
>   spin_lock_irq(lock);
> - if (!q->mq_ops)
> - __blk_drain_queue(q, true);
>   queue_flag_set(QUEUE_FLAG_DEAD, q);
>   spin_unlock_irq(lock);
>  
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 11097477eeab..3d3797327491 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -161,6 +161,8 @@ void blk_freeze_queue(struct request_queue *q)
>* exported to drivers as the only user for unfreeze is blk_mq.
>*/
>   blk_freeze_queue_start(q);
> + if (!q->mq_ops)
> + blk_drain_queue(q);
>   blk_mq_freeze_queue_wait(q);
>  }
>  
> diff --git a/block/blk.h b/block/blk.h
> index 3f1446937aec..442098aa9463 100644
> --- a/block/blk.h
> +++ b/block/blk.h
> @@ -330,4 +330,6 @@ static inline void blk_queue_bounce(struct request_queue 
> *q, struct bio **bio)
>  }
>  #endif /* CONFIG_BOUNCE */
>  
> +extern void blk_drain_queue(struct request_queue *q);
> +
>  #endif /* BLK_INTERNAL_H */
> 
I seem to have missed something.
Is blk_drain_queue() ever used?

Cheers,

Hannes
-- 
Dr. Hannes ReineckeTeamlead Storage & Networking
h...@suse.de   +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH] block: drain queue before waiting for q_usage_counter becoming zero

2017-11-21 Thread Ming Lei 
Now we track legacy requests with .q_usage_counter in commit 055f6e18e08f
("block: Make q_usage_counter also track legacy requests"), but that
commit never runs and drains legacy queue before waiting for this counter
becoming zero, then IO hang is caused in the test of pulling disk during IO.

This patch fixes the issue by draining requests before waiting for
q_usage_counter becoming zero.

Fixes: 055f6e18e08f("block: Make q_usage_counter also track legacy requests")
Cc: Wen Xiong 
Signed-off-by: Ming Lei 
---
 block/blk-core.c | 9 +++--
 block/blk-mq.c   | 2 ++
 block/blk.h  | 2 ++
 3 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 1038706edd87..5ae5ea0dca4c 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -562,6 +562,13 @@ static void __blk_drain_queue(struct request_queue *q, 
bool drain_all)
}
 }
 
+void blk_drain_queue(struct request_queue *q)
+{
+   spin_lock_irq(q->queue_lock);
+   __blk_drain_queue(q, true);
+   spin_unlock_irq(q->queue_lock);
+}
+
 /**
  * blk_queue_bypass_start - enter queue bypass mode
  * @q: queue of interest
@@ -689,8 +696,6 @@ void blk_cleanup_queue(struct request_queue *q)
 */
blk_freeze_queue(q);
spin_lock_irq(lock);
-   if (!q->mq_ops)
-   __blk_drain_queue(q, true);
queue_flag_set(QUEUE_FLAG_DEAD, q);
spin_unlock_irq(lock);
 
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 11097477eeab..3d3797327491 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -161,6 +161,8 @@ void blk_freeze_queue(struct request_queue *q)
 * exported to drivers as the only user for unfreeze is blk_mq.
 */
blk_freeze_queue_start(q);
+   if (!q->mq_ops)
+   blk_drain_queue(q);
blk_mq_freeze_queue_wait(q);
 }
 
diff --git a/block/blk.h b/block/blk.h
index 3f1446937aec..442098aa9463 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -330,4 +330,6 @@ static inline void blk_queue_bounce(struct request_queue 
*q, struct bio **bio)
 }
 #endif /* CONFIG_BOUNCE */
 
+extern void blk_drain_queue(struct request_queue *q);
+
 #endif /* BLK_INTERNAL_H */
-- 
2.9.5