Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
 Hello, guys.
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:

I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
I guess this is possible. It is often only a single request
that is on the dispatch list though, so I don't know if it
would make sense to reorder it by priority again.

Depends entirely on the io scheduler. CFQ may put several on the
dispatch list.

on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.
I've thought it was SOFTBARRIER ensures the device driver (and
hardware?) sees the request in this order, and HARDBARRIER ensures
it reaches stable storage in this order.
Not exactly sure why you would want a softbarrier and not a
hardbarrier. Maybe for special commands.

It is the cleaner interpretation. CFQ marks requests as requeued
internally and gives preference to them for reissue, but it may return
another first (actually, I think it even checks for ->requeued on
dispatch sort, so it wont right now).

I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.
OK - but I'm wondering would it ever make sense to do it any
other way? I would have thought no, in which case we can document
that requests seen by 'elv_next_request', and those requeued back
into the device will not be reordered, and so Tejun does not need
to set REQ_SOFTBARRIER.
But I'm not so sure now... it isn't really that big a deal ;)
So whatever you're happy with is fine. Sorry for the nose.

It's not noise, it would be nice to have this entirely documented so
that there isn't any confusion on what is guaranteed vs what currently
happens in most places.
But I don't want to document that they are never reordered. For requeues
it make sense to maintain ordering in most cases, but it also may make
sense to reorder for higher priority io. If the driver does _not_ want a
particular request reordered for data integrity reasons, then that needs
to be explicitly specified.
 So, I guess this is settled now.  James, what do you think about the 
rest of this patchset?  If you're okay, I think we can proceed merging 
as there doesn't seem to be any issue left.  Do we put this into the 
SCSI tree?  Or separate out blk and SCSI changes?

 Once we're done merging this patchset, I'll regenerate & repost the 
reqfn reimpl patchset.

 Thanks a lot. :-)
--
tejun
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Nick Piggin wrote:
> Jens Axboe wrote:
> >On Wed, Apr 20 2005, Nick Piggin wrote:
> >
> 
> >>I guess this could be one use of 'reordering' after a requeue.
> >
> >
> >Yeah, or perhaps the io scheduler might determine that a request has
> >higher prio than a requeued one.  I'm not sure what semantics to place
> 
> I guess this is possible. It is often only a single request
> that is on the dispatch list though, so I don't know if it
> would make sense to reorder it by priority again.

Depends entirely on the io scheduler. CFQ may put several on the
dispatch list.

> >on soft-barrier, I've always taken it to mean 'maintain ordering if
> >convenient' where the hard-barrier must be followed.
> >
> 
> I've thought it was SOFTBARRIER ensures the device driver (and
> hardware?) sees the request in this order, and HARDBARRIER ensures
> it reaches stable storage in this order.
> 
> Not exactly sure why you would want a softbarrier and not a
> hardbarrier. Maybe for special commands.

It is the cleaner interpretation. CFQ marks requests as requeued
internally and gives preference to them for reissue, but it may return
another first (actually, I think it even checks for ->requeued on
dispatch sort, so it wont right now).

> >>I'm not sure this would need a REQ_SOFTBARRIER either though, really.
> >>
> >>Your basic io scheduler framework - ie. a FIFO dispatch list which
> >>can have requests requeued on the front models pretty well what the
> >>block layer needs of the elevator.
> >>
> >>Considering all requeues and all elv_next_request but not dequeued
> >>requests would have this REQ_SOFTBARRIER bit set, any other model
> >>that theoretically would allow reordering would degenerate to this
> >>dispatch list behaviour, right?
> >
> >
> >Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
> >on elv_next_request() return, it should only happen on requeues.
> >REQ_STARTED implies that you should not pass this request, since the io
> >scheduler is required to return this request again until dequeue is
> >called. But the result is the same, correct.
> >
> 
> OK - but I'm wondering would it ever make sense to do it any
> other way? I would have thought no, in which case we can document
> that requests seen by 'elv_next_request', and those requeued back
> into the device will not be reordered, and so Tejun does not need
> to set REQ_SOFTBARRIER.
> 
> But I'm not so sure now... it isn't really that big a deal ;)
> So whatever you're happy with is fine. Sorry for the nose.

It's not noise, it would be nice to have this entirely documented so
that there isn't any confusion on what is guaranteed vs what currently
happens in most places.

But I don't want to document that they are never reordered. For requeues
it make sense to maintain ordering in most cases, but it also may make
sense to reorder for higher priority io. If the driver does _not_ want a
particular request reordered for data integrity reasons, then that needs
to be explicitly specified.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:

I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
I guess this is possible. It is often only a single request
that is on the dispatch list though, so I don't know if it
would make sense to reorder it by priority again.
on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.
I've thought it was SOFTBARRIER ensures the device driver (and
hardware?) sees the request in this order, and HARDBARRIER ensures
it reaches stable storage in this order.
Not exactly sure why you would want a softbarrier and not a
hardbarrier. Maybe for special commands.

I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.
OK - but I'm wondering would it ever make sense to do it any
other way? I would have thought no, in which case we can document
that requests seen by 'elv_next_request', and those requeued back
into the device will not be reordered, and so Tejun does not need
to set REQ_SOFTBARRIER.
But I'm not so sure now... it isn't really that big a deal ;)
So whatever you're happy with is fine. Sorry for the nose.
--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Nick Piggin wrote:
> >>Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
> >>necessary as we have INSERT_FRONT policy on requeue, and if
> >>elv_next_req_fn() is required to return the same request when the
> >>request isn't dequeued, you're right and we don't need this patch at
> >>all.  We are guaranteed that all requeued requests are served in LIFO
> >>manner.
> >
> >
> >After a requeue, it is not required to return the same request again.
> >
> 
> Well I guess not.
> 
> Would there be any benefit to reordering after a requeue?

Logic dictates that requeues should maintain ordering, since we don't
want to reorder around the original io scheduler decisions. But you
could be requeuing more than one request.

> >>BTW, the same un-dequeued request rule is sort of already broken as
> >>INSERT_FRONT request passes a returned but un-dequeued request, but,
> >>then again, we need this behavior as we have to favor fully-prepped
> >>requests over partially-prepped one.
> >
> >
> >INSERT_FRONT really should skip requests with REQ_STARTED on the
> >dispatch list to be fully safe.
> >
> 
> I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.

> I'm not sure this would need a REQ_SOFTBARRIER either though, really.
> 
> Your basic io scheduler framework - ie. a FIFO dispatch list which
> can have requests requeued on the front models pretty well what the
> block layer needs of the elevator.
> 
> Considering all requeues and all elv_next_request but not dequeued
> requests would have this REQ_SOFTBARRIER bit set, any other model
> that theoretically would allow reordering would degenerate to this
> dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.

> In which case, the dispatch list is effectively basically the most
> efficient way to do it? In which case should we just explicitly
> document that in the API?

You lost me, please detail exactly what behaviour you want documented :)

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
Jens Axboe wrote:
On Wed, Apr 20 2005, Tejun Heo wrote:

Well, yeah, all schedulers have dispatch queue (noop has only the
dispatch queue) and use them to defer/requeue, so no reordering will
happen, but I'm not sure they are required to be like this or just
happen to be implemented so.

Precisely, I feel much better making sure SOFTBARRIER is set so that we
_know_ that a scheduler following the outlined rules will do the right
thing.
Well yeah, at the moment I am just following implementations as
defining the standard.

Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
necessary as we have INSERT_FRONT policy on requeue, and if
elv_next_req_fn() is required to return the same request when the
request isn't dequeued, you're right and we don't need this patch at
all.  We are guaranteed that all requeued requests are served in LIFO
manner.

After a requeue, it is not required to return the same request again.
Well I guess not.
Would there be any benefit to reordering after a requeue?

BTW, the same un-dequeued request rule is sort of already broken as
INSERT_FRONT request passes a returned but un-dequeued request, but,
then again, we need this behavior as we have to favor fully-prepped
requests over partially-prepped one.

INSERT_FRONT really should skip requests with REQ_STARTED on the
dispatch list to be fully safe.
I guess this could be one use of 'reordering' after a requeue.
I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?
In which case, the dispatch list is effectively basically the most
efficient way to do it? In which case should we just explicitly
document that in the API?
--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Tejun Heo wrote:
> Nick Piggin wrote:
> > On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
> > 
> >> Hello, Jens.
> >>
> >>On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
> >>
> >>>Do it on requeue, please - not on the initial spotting of the request.
> >>
> >> This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
> >>in two places - in elv_next_request() on BLKPREP_DEFER and in
> >>blk_requeue_request().
> >>
> >> Other patches apply cleanly with this patch or the original one and
> >>the end result is the same, so take your pick.  :-)
> >>
> > 
> > 
> > I'm not sure that you need *either* one.
> > 
> > As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
> > into the top of the block layer, and is used to guarantee the device
> > driver gets the requests in a specific ordering.
> > 
> > When dealing with the requests at the other end (ie.
> > elevator_next_req_fn, blk_requeue_request), then ordering does not
> > change.
> > 
> > That is - if you call elevator_next_req_fn and don't dequeue the
> > request, then that's the same request you'll get next time.
> > 
> > And blk_requeue_request will push the request back onto the end of
> > the queue in a LIFO manner.
> > 
> > So I think adding barriers, apart from not doing anything, confuses
> > the issue because it suggests there *could* be reordering without
> > them.
> > 
> > Or am I completely wrong? It's been a while since I last got into
> > the code.
> 
>  Well, yeah, all schedulers have dispatch queue (noop has only the
> dispatch queue) and use them to defer/requeue, so no reordering will
> happen, but I'm not sure they are required to be like this or just
> happen to be implemented so.

Precisely, I feel much better making sure SOFTBARRIER is set so that we
_know_ that a scheduler following the outlined rules will do the right
thing.

>  Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
> necessary as we have INSERT_FRONT policy on requeue, and if
> elv_next_req_fn() is required to return the same request when the
> request isn't dequeued, you're right and we don't need this patch at
> all.  We are guaranteed that all requeued requests are served in LIFO
> manner.

After a requeue, it is not required to return the same request again.

>  BTW, the same un-dequeued request rule is sort of already broken as
> INSERT_FRONT request passes a returned but un-dequeued request, but,
> then again, we need this behavior as we have to favor fully-prepped
> requests over partially-prepped one.

INSERT_FRONT really should skip requests with REQ_STARTED on the
dispatch list to be fully safe.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
Nick Piggin wrote:
> On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
> 
>> Hello, Jens.
>>
>>On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
>>
>>>Do it on requeue, please - not on the initial spotting of the request.
>>
>> This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
>>in two places - in elv_next_request() on BLKPREP_DEFER and in
>>blk_requeue_request().
>>
>> Other patches apply cleanly with this patch or the original one and
>>the end result is the same, so take your pick.  :-)
>>
> 
> 
> I'm not sure that you need *either* one.
> 
> As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
> into the top of the block layer, and is used to guarantee the device
> driver gets the requests in a specific ordering.
> 
> When dealing with the requests at the other end (ie.
> elevator_next_req_fn, blk_requeue_request), then ordering does not
> change.
> 
> That is - if you call elevator_next_req_fn and don't dequeue the
> request, then that's the same request you'll get next time.
> 
> And blk_requeue_request will push the request back onto the end of
> the queue in a LIFO manner.
> 
> So I think adding barriers, apart from not doing anything, confuses
> the issue because it suggests there *could* be reordering without
> them.
> 
> Or am I completely wrong? It's been a while since I last got into
> the code.

 Well, yeah, all schedulers have dispatch queue (noop has only the
dispatch queue) and use them to defer/requeue, so no reordering will
happen, but I'm not sure they are required to be like this or just
happen to be implemented so.

 Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
necessary as we have INSERT_FRONT policy on requeue, and if
elv_next_req_fn() is required to return the same request when the
request isn't dequeued, you're right and we don't need this patch at
all.  We are guaranteed that all requeued requests are served in LIFO
manner.

 BTW, the same un-dequeued request rule is sort of already broken as
INSERT_FRONT request passes a returned but un-dequeued request, but,
then again, we need this behavior as we have to favor fully-prepped
requests over partially-prepped one.

-- 
tejun

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
>  Hello, Jens.
> 
> On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
> > Do it on requeue, please - not on the initial spotting of the request.
> 
>  This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
> in two places - in elv_next_request() on BLKPREP_DEFER and in
> blk_requeue_request().
> 
>  Other patches apply cleanly with this patch or the original one and
> the end result is the same, so take your pick.  :-)
> 

I'm not sure that you need *either* one.

As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
into the top of the block layer, and is used to guarantee the device
driver gets the requests in a specific ordering.

When dealing with the requests at the other end (ie.
elevator_next_req_fn, blk_requeue_request), then ordering does not
change.

That is - if you call elevator_next_req_fn and don't dequeue the
request, then that's the same request you'll get next time.

And blk_requeue_request will push the request back onto the end of
the queue in a LIFO manner.

So I think adding barriers, apart from not doing anything, confuses
the issue because it suggests there *could* be reordering without
them.

Or am I completely wrong? It's been a while since I last got into
the code.


-- 
SUSE Labs, Novell Inc.



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
 Hello, Jens.

On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
> Do it on requeue, please - not on the initial spotting of the request.

 This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
in two places - in elv_next_request() on BLKPREP_DEFER and in
blk_requeue_request().

 Other patches apply cleanly with this patch or the original one and
the end result is the same, so take your pick.  :-)


 Signed-off-by: Tejun Heo <[EMAIL PROTECTED]>


Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
16:24:26.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 16:31:36.0 
+0900
@@ -291,6 +291,13 @@ void elv_requeue_request(request_queue_t
}
 
/*
+* the request is prepped and may have some resources allocated.
+* allowing unprepped requests to pass this one may cause resource
+* deadlock.  turn on softbarrier.
+*/
+   rq->flags |= REQ_SOFTBARRIER;
+
+   /*
 * if iosched has an explicit requeue hook, then use that. otherwise
 * just put the request at the front of the queue
 */
@@ -386,6 +393,12 @@ struct request *elv_next_request(request
if (ret == BLKPREP_OK) {
break;
} else if (ret == BLKPREP_DEFER) {
+   /*
+* the request may have been (partially) prepped.
+* we need to keep this request in the front to
+* avoid resource deadlock.  turn on softbarrier.
+*/
+   rq->flags |= REQ_SOFTBARRIER;
rq = NULL;
break;
} else if (ret == BLKPREP_KILL) {
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
Jens Axboe wrote:
On Wed, Apr 20 2005, Tejun Heo wrote:
01_scsi_blk_make_started_requests_ordered.patch
Reordering already started requests is without any real
benefit and causes problems if the request has its
driver-specific resources allocated (as in SCSI).  This patch
makes elv_next_request() set REQ_SOFTBARRIER automatically
when a request is dispatched.
As both as and cfq schedulers don't allow passing requeued
requests, the only behavior change is that requests deferred
by prep_fn won't be passed by other requests.  This change
shouldn't cause any problem.  The only affected driver other
than SCSI is i2o_block.
Signed-off-by: Tejun Heo <[EMAIL PROTECTED]>
elevator.c |8 
1 files changed, 4 insertions(+), 4 deletions(-)
Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
08:13:01.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 08:13:33.0 
+0900
@@ -370,11 +370,11 @@ struct request *elv_next_request(request
while ((rq = __elv_next_request(q)) != NULL) {
/*
-* just mark as started even if we don't start it, a request
-* that has been delayed should not be passed by new incoming
-* requests
+* just mark as started even if we don't start it.
+* also, as a request that has been delayed should not
+* be passed by new incoming requests, set softbarrier.
 */
-   rq->flags |= REQ_STARTED;
+   rq->flags |= REQ_STARTED | REQ_SOFTBARRIER;
if (rq == q->last_merge)
q->last_merge = NULL;

Do it on requeue, please - not on the initial spotting of the request.
 The thing is that we also need to set REQ_SOFTBARRIER on 
BLKPREP_DEFER.  So, it will be two places - in elv_next_request and 
blk_requeue_request.  The end result will be the same.  Do you think 
doing on requeue paths is better?

--
tejun
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Tejun Heo wrote:
> 01_scsi_blk_make_started_requests_ordered.patch
> 
>   Reordering already started requests is without any real
>   benefit and causes problems if the request has its
>   driver-specific resources allocated (as in SCSI).  This patch
>   makes elv_next_request() set REQ_SOFTBARRIER automatically
>   when a request is dispatched.
> 
>   As both as and cfq schedulers don't allow passing requeued
>   requests, the only behavior change is that requests deferred
>   by prep_fn won't be passed by other requests.  This change
>   shouldn't cause any problem.  The only affected driver other
>   than SCSI is i2o_block.
> 
> Signed-off-by: Tejun Heo <[EMAIL PROTECTED]>
> 
>  elevator.c |8 
>  1 files changed, 4 insertions(+), 4 deletions(-)
> 
> Index: scsi-reqfn-export/drivers/block/elevator.c
> ===
> --- scsi-reqfn-export.orig/drivers/block/elevator.c   2005-04-20 
> 08:13:01.0 +0900
> +++ scsi-reqfn-export/drivers/block/elevator.c2005-04-20 
> 08:13:33.0 +0900
> @@ -370,11 +370,11 @@ struct request *elv_next_request(request
>  
>   while ((rq = __elv_next_request(q)) != NULL) {
>   /*
> -  * just mark as started even if we don't start it, a request
> -  * that has been delayed should not be passed by new incoming
> -  * requests
> +  * just mark as started even if we don't start it.
> +  * also, as a request that has been delayed should not
> +  * be passed by new incoming requests, set softbarrier.
>*/
> - rq->flags |= REQ_STARTED;
> + rq->flags |= REQ_STARTED | REQ_SOFTBARRIER;
>  
>   if (rq == q->last_merge)
>   q->last_merge = NULL;

Do it on requeue, please - not on the initial spotting of the request.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Tejun Heo wrote:
 01_scsi_blk_make_started_requests_ordered.patch
 
   Reordering already started requests is without any real
   benefit and causes problems if the request has its
   driver-specific resources allocated (as in SCSI).  This patch
   makes elv_next_request() set REQ_SOFTBARRIER automatically
   when a request is dispatched.
 
   As both as and cfq schedulers don't allow passing requeued
   requests, the only behavior change is that requests deferred
   by prep_fn won't be passed by other requests.  This change
   shouldn't cause any problem.  The only affected driver other
   than SCSI is i2o_block.
 
 Signed-off-by: Tejun Heo [EMAIL PROTECTED]
 
  elevator.c |8 
  1 files changed, 4 insertions(+), 4 deletions(-)
 
 Index: scsi-reqfn-export/drivers/block/elevator.c
 ===
 --- scsi-reqfn-export.orig/drivers/block/elevator.c   2005-04-20 
 08:13:01.0 +0900
 +++ scsi-reqfn-export/drivers/block/elevator.c2005-04-20 
 08:13:33.0 +0900
 @@ -370,11 +370,11 @@ struct request *elv_next_request(request
  
   while ((rq = __elv_next_request(q)) != NULL) {
   /*
 -  * just mark as started even if we don't start it, a request
 -  * that has been delayed should not be passed by new incoming
 -  * requests
 +  * just mark as started even if we don't start it.
 +  * also, as a request that has been delayed should not
 +  * be passed by new incoming requests, set softbarrier.
*/
 - rq-flags |= REQ_STARTED;
 + rq-flags |= REQ_STARTED | REQ_SOFTBARRIER;
  
   if (rq == q-last_merge)
   q-last_merge = NULL;

Do it on requeue, please - not on the initial spotting of the request.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
Jens Axboe wrote:
On Wed, Apr 20 2005, Tejun Heo wrote:
01_scsi_blk_make_started_requests_ordered.patch
Reordering already started requests is without any real
benefit and causes problems if the request has its
driver-specific resources allocated (as in SCSI).  This patch
makes elv_next_request() set REQ_SOFTBARRIER automatically
when a request is dispatched.
As both as and cfq schedulers don't allow passing requeued
requests, the only behavior change is that requests deferred
by prep_fn won't be passed by other requests.  This change
shouldn't cause any problem.  The only affected driver other
than SCSI is i2o_block.
Signed-off-by: Tejun Heo [EMAIL PROTECTED]
elevator.c |8 
1 files changed, 4 insertions(+), 4 deletions(-)
Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
08:13:01.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 08:13:33.0 
+0900
@@ -370,11 +370,11 @@ struct request *elv_next_request(request
while ((rq = __elv_next_request(q)) != NULL) {
/*
-* just mark as started even if we don't start it, a request
-* that has been delayed should not be passed by new incoming
-* requests
+* just mark as started even if we don't start it.
+* also, as a request that has been delayed should not
+* be passed by new incoming requests, set softbarrier.
 */
-   rq-flags |= REQ_STARTED;
+   rq-flags |= REQ_STARTED | REQ_SOFTBARRIER;
if (rq == q-last_merge)
q-last_merge = NULL;

Do it on requeue, please - not on the initial spotting of the request.
 The thing is that we also need to set REQ_SOFTBARRIER on 
BLKPREP_DEFER.  So, it will be two places - in elv_next_request and 
blk_requeue_request.  The end result will be the same.  Do you think 
doing on requeue paths is better?

--
tejun
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
 Hello, Jens.

On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
 Do it on requeue, please - not on the initial spotting of the request.

 This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
in two places - in elv_next_request() on BLKPREP_DEFER and in
blk_requeue_request().

 Other patches apply cleanly with this patch or the original one and
the end result is the same, so take your pick.  :-)


 Signed-off-by: Tejun Heo [EMAIL PROTECTED]


Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
16:24:26.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 16:31:36.0 
+0900
@@ -291,6 +291,13 @@ void elv_requeue_request(request_queue_t
}
 
/*
+* the request is prepped and may have some resources allocated.
+* allowing unprepped requests to pass this one may cause resource
+* deadlock.  turn on softbarrier.
+*/
+   rq-flags |= REQ_SOFTBARRIER;
+
+   /*
 * if iosched has an explicit requeue hook, then use that. otherwise
 * just put the request at the front of the queue
 */
@@ -386,6 +393,12 @@ struct request *elv_next_request(request
if (ret == BLKPREP_OK) {
break;
} else if (ret == BLKPREP_DEFER) {
+   /*
+* the request may have been (partially) prepped.
+* we need to keep this request in the front to
+* avoid resource deadlock.  turn on softbarrier.
+*/
+   rq-flags |= REQ_SOFTBARRIER;
rq = NULL;
break;
} else if (ret == BLKPREP_KILL) {
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
  Hello, Jens.
 
 On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
  Do it on requeue, please - not on the initial spotting of the request.
 
  This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
 in two places - in elv_next_request() on BLKPREP_DEFER and in
 blk_requeue_request().
 
  Other patches apply cleanly with this patch or the original one and
 the end result is the same, so take your pick.  :-)
 

I'm not sure that you need *either* one.

As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
into the top of the block layer, and is used to guarantee the device
driver gets the requests in a specific ordering.

When dealing with the requests at the other end (ie.
elevator_next_req_fn, blk_requeue_request), then ordering does not
change.

That is - if you call elevator_next_req_fn and don't dequeue the
request, then that's the same request you'll get next time.

And blk_requeue_request will push the request back onto the end of
the queue in a LIFO manner.

So I think adding barriers, apart from not doing anything, confuses
the issue because it suggests there *could* be reordering without
them.

Or am I completely wrong? It's been a while since I last got into
the code.


-- 
SUSE Labs, Novell Inc.



-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
Nick Piggin wrote:
 On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
 
 Hello, Jens.

On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:

Do it on requeue, please - not on the initial spotting of the request.

 This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
in two places - in elv_next_request() on BLKPREP_DEFER and in
blk_requeue_request().

 Other patches apply cleanly with this patch or the original one and
the end result is the same, so take your pick.  :-)

 
 
 I'm not sure that you need *either* one.
 
 As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
 into the top of the block layer, and is used to guarantee the device
 driver gets the requests in a specific ordering.
 
 When dealing with the requests at the other end (ie.
 elevator_next_req_fn, blk_requeue_request), then ordering does not
 change.
 
 That is - if you call elevator_next_req_fn and don't dequeue the
 request, then that's the same request you'll get next time.
 
 And blk_requeue_request will push the request back onto the end of
 the queue in a LIFO manner.
 
 So I think adding barriers, apart from not doing anything, confuses
 the issue because it suggests there *could* be reordering without
 them.
 
 Or am I completely wrong? It's been a while since I last got into
 the code.

 Well, yeah, all schedulers have dispatch queue (noop has only the
dispatch queue) and use them to defer/requeue, so no reordering will
happen, but I'm not sure they are required to be like this or just
happen to be implemented so.

 Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
necessary as we have INSERT_FRONT policy on requeue, and if
elv_next_req_fn() is required to return the same request when the
request isn't dequeued, you're right and we don't need this patch at
all.  We are guaranteed that all requeued requests are served in LIFO
manner.

 BTW, the same un-dequeued request rule is sort of already broken as
INSERT_FRONT request passes a returned but un-dequeued request, but,
then again, we need this behavior as we have to favor fully-prepped
requests over partially-prepped one.

-- 
tejun

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Tejun Heo wrote:
 Nick Piggin wrote:
  On Wed, 2005-04-20 at 16:40 +0900, Tejun Heo wrote:
  
  Hello, Jens.
 
 On Wed, Apr 20, 2005 at 08:30:10AM +0200, Jens Axboe wrote:
 
 Do it on requeue, please - not on the initial spotting of the request.
 
  This is the reworked version of the patch.  It sets REQ_SOFTBARRIER
 in two places - in elv_next_request() on BLKPREP_DEFER and in
 blk_requeue_request().
 
  Other patches apply cleanly with this patch or the original one and
 the end result is the same, so take your pick.  :-)
 
  
  
  I'm not sure that you need *either* one.
  
  As far as I'm aware, REQ_SOFTBARRIER is used when feeding requests
  into the top of the block layer, and is used to guarantee the device
  driver gets the requests in a specific ordering.
  
  When dealing with the requests at the other end (ie.
  elevator_next_req_fn, blk_requeue_request), then ordering does not
  change.
  
  That is - if you call elevator_next_req_fn and don't dequeue the
  request, then that's the same request you'll get next time.
  
  And blk_requeue_request will push the request back onto the end of
  the queue in a LIFO manner.
  
  So I think adding barriers, apart from not doing anything, confuses
  the issue because it suggests there *could* be reordering without
  them.
  
  Or am I completely wrong? It's been a while since I last got into
  the code.
 
  Well, yeah, all schedulers have dispatch queue (noop has only the
 dispatch queue) and use them to defer/requeue, so no reordering will
 happen, but I'm not sure they are required to be like this or just
 happen to be implemented so.

Precisely, I feel much better making sure SOFTBARRIER is set so that we
_know_ that a scheduler following the outlined rules will do the right
thing.

  Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
 necessary as we have INSERT_FRONT policy on requeue, and if
 elv_next_req_fn() is required to return the same request when the
 request isn't dequeued, you're right and we don't need this patch at
 all.  We are guaranteed that all requeued requests are served in LIFO
 manner.

After a requeue, it is not required to return the same request again.

  BTW, the same un-dequeued request rule is sort of already broken as
 INSERT_FRONT request passes a returned but un-dequeued request, but,
 then again, we need this behavior as we have to favor fully-prepped
 requests over partially-prepped one.

INSERT_FRONT really should skip requests with REQ_STARTED on the
dispatch list to be fully safe.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
Jens Axboe wrote:
On Wed, Apr 20 2005, Tejun Heo wrote:

Well, yeah, all schedulers have dispatch queue (noop has only the
dispatch queue) and use them to defer/requeue, so no reordering will
happen, but I'm not sure they are required to be like this or just
happen to be implemented so.

Precisely, I feel much better making sure SOFTBARRIER is set so that we
_know_ that a scheduler following the outlined rules will do the right
thing.
Well yeah, at the moment I am just following implementations as
defining the standard.

Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
necessary as we have INSERT_FRONT policy on requeue, and if
elv_next_req_fn() is required to return the same request when the
request isn't dequeued, you're right and we don't need this patch at
all.  We are guaranteed that all requeued requests are served in LIFO
manner.

After a requeue, it is not required to return the same request again.
Well I guess not.
Would there be any benefit to reordering after a requeue?

BTW, the same un-dequeued request rule is sort of already broken as
INSERT_FRONT request passes a returned but un-dequeued request, but,
then again, we need this behavior as we have to favor fully-prepped
requests over partially-prepped one.

INSERT_FRONT really should skip requests with REQ_STARTED on the
dispatch list to be fully safe.
I guess this could be one use of 'reordering' after a requeue.
I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?
In which case, the dispatch list is effectively basically the most
efficient way to do it? In which case should we just explicitly
document that in the API?
--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Nick Piggin wrote:
 Hmm, well, it seems that setting REQ_SOFTBARRIER on requeue path isn't
 necessary as we have INSERT_FRONT policy on requeue, and if
 elv_next_req_fn() is required to return the same request when the
 request isn't dequeued, you're right and we don't need this patch at
 all.  We are guaranteed that all requeued requests are served in LIFO
 manner.
 
 
 After a requeue, it is not required to return the same request again.
 
 
 Well I guess not.
 
 Would there be any benefit to reordering after a requeue?

Logic dictates that requeues should maintain ordering, since we don't
want to reorder around the original io scheduler decisions. But you
could be requeuing more than one request.

 BTW, the same un-dequeued request rule is sort of already broken as
 INSERT_FRONT request passes a returned but un-dequeued request, but,
 then again, we need this behavior as we have to favor fully-prepped
 requests over partially-prepped one.
 
 
 INSERT_FRONT really should skip requests with REQ_STARTED on the
 dispatch list to be fully safe.
 
 
 I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.

 I'm not sure this would need a REQ_SOFTBARRIER either though, really.
 
 Your basic io scheduler framework - ie. a FIFO dispatch list which
 can have requests requeued on the front models pretty well what the
 block layer needs of the elevator.
 
 Considering all requeues and all elv_next_request but not dequeued
 requests would have this REQ_SOFTBARRIER bit set, any other model
 that theoretically would allow reordering would degenerate to this
 dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.

 In which case, the dispatch list is effectively basically the most
 efficient way to do it? In which case should we just explicitly
 document that in the API?

You lost me, please detail exactly what behaviour you want documented :)

-- 
Jens Axboe

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Nick Piggin
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:

I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
I guess this is possible. It is often only a single request
that is on the dispatch list though, so I don't know if it
would make sense to reorder it by priority again.
on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.
I've thought it was SOFTBARRIER ensures the device driver (and
hardware?) sees the request in this order, and HARDBARRIER ensures
it reaches stable storage in this order.
Not exactly sure why you would want a softbarrier and not a
hardbarrier. Maybe for special commands.

I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.
OK - but I'm wondering would it ever make sense to do it any
other way? I would have thought no, in which case we can document
that requests seen by 'elv_next_request', and those requeued back
into the device will not be reordered, and so Tejun does not need
to set REQ_SOFTBARRIER.
But I'm not so sure now... it isn't really that big a deal ;)
So whatever you're happy with is fine. Sorry for the nose.
--
SUSE Labs, Novell Inc.
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Jens Axboe
On Wed, Apr 20 2005, Nick Piggin wrote:
 Jens Axboe wrote:
 On Wed, Apr 20 2005, Nick Piggin wrote:
 
 
 I guess this could be one use of 'reordering' after a requeue.
 
 
 Yeah, or perhaps the io scheduler might determine that a request has
 higher prio than a requeued one.  I'm not sure what semantics to place
 
 I guess this is possible. It is often only a single request
 that is on the dispatch list though, so I don't know if it
 would make sense to reorder it by priority again.

Depends entirely on the io scheduler. CFQ may put several on the
dispatch list.

 on soft-barrier, I've always taken it to mean 'maintain ordering if
 convenient' where the hard-barrier must be followed.
 
 
 I've thought it was SOFTBARRIER ensures the device driver (and
 hardware?) sees the request in this order, and HARDBARRIER ensures
 it reaches stable storage in this order.
 
 Not exactly sure why you would want a softbarrier and not a
 hardbarrier. Maybe for special commands.

It is the cleaner interpretation. CFQ marks requests as requeued
internally and gives preference to them for reissue, but it may return
another first (actually, I think it even checks for -requeued on
dispatch sort, so it wont right now).

 I'm not sure this would need a REQ_SOFTBARRIER either though, really.
 
 Your basic io scheduler framework - ie. a FIFO dispatch list which
 can have requests requeued on the front models pretty well what the
 block layer needs of the elevator.
 
 Considering all requeues and all elv_next_request but not dequeued
 requests would have this REQ_SOFTBARRIER bit set, any other model
 that theoretically would allow reordering would degenerate to this
 dispatch list behaviour, right?
 
 
 Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
 on elv_next_request() return, it should only happen on requeues.
 REQ_STARTED implies that you should not pass this request, since the io
 scheduler is required to return this request again until dequeue is
 called. But the result is the same, correct.
 
 
 OK - but I'm wondering would it ever make sense to do it any
 other way? I would have thought no, in which case we can document
 that requests seen by 'elv_next_request', and those requeued back
 into the device will not be reordered, and so Tejun does not need
 to set REQ_SOFTBARRIER.
 
 But I'm not so sure now... it isn't really that big a deal ;)
 So whatever you're happy with is fine. Sorry for the nose.

It's not noise, it would be nice to have this entirely documented so
that there isn't any confusion on what is guaranteed vs what currently
happens in most places.

But I don't want to document that they are never reordered. For requeues
it make sense to maintain ordering in most cases, but it also may make
sense to reorder for higher priority io. If the driver does _not_ want a
particular request reordered for data integrity reasons, then that needs
to be explicitly specified.

-- 
Jens Axboe

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-20 Thread Tejun Heo
 Hello, guys.
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:
Jens Axboe wrote:
On Wed, Apr 20 2005, Nick Piggin wrote:

I guess this could be one use of 'reordering' after a requeue.

Yeah, or perhaps the io scheduler might determine that a request has
higher prio than a requeued one.  I'm not sure what semantics to place
I guess this is possible. It is often only a single request
that is on the dispatch list though, so I don't know if it
would make sense to reorder it by priority again.

Depends entirely on the io scheduler. CFQ may put several on the
dispatch list.

on soft-barrier, I've always taken it to mean 'maintain ordering if
convenient' where the hard-barrier must be followed.
I've thought it was SOFTBARRIER ensures the device driver (and
hardware?) sees the request in this order, and HARDBARRIER ensures
it reaches stable storage in this order.
Not exactly sure why you would want a softbarrier and not a
hardbarrier. Maybe for special commands.

It is the cleaner interpretation. CFQ marks requests as requeued
internally and gives preference to them for reissue, but it may return
another first (actually, I think it even checks for -requeued on
dispatch sort, so it wont right now).

I'm not sure this would need a REQ_SOFTBARRIER either though, really.
Your basic io scheduler framework - ie. a FIFO dispatch list which
can have requests requeued on the front models pretty well what the
block layer needs of the elevator.
Considering all requeues and all elv_next_request but not dequeued
requests would have this REQ_SOFTBARRIER bit set, any other model
that theoretically would allow reordering would degenerate to this
dispatch list behaviour, right?

Not sure I follow this - I don't want REQ_SOFTBARRIER set automatically
on elv_next_request() return, it should only happen on requeues.
REQ_STARTED implies that you should not pass this request, since the io
scheduler is required to return this request again until dequeue is
called. But the result is the same, correct.
OK - but I'm wondering would it ever make sense to do it any
other way? I would have thought no, in which case we can document
that requests seen by 'elv_next_request', and those requeued back
into the device will not be reordered, and so Tejun does not need
to set REQ_SOFTBARRIER.
But I'm not so sure now... it isn't really that big a deal ;)
So whatever you're happy with is fine. Sorry for the nose.

It's not noise, it would be nice to have this entirely documented so
that there isn't any confusion on what is guaranteed vs what currently
happens in most places.
But I don't want to document that they are never reordered. For requeues
it make sense to maintain ordering in most cases, but it also may make
sense to reorder for higher priority io. If the driver does _not_ want a
particular request reordered for data integrity reasons, then that needs
to be explicitly specified.
 So, I guess this is settled now.  James, what do you think about the 
rest of this patchset?  If you're okay, I think we can proceed merging 
as there doesn't seem to be any issue left.  Do we put this into the 
SCSI tree?  Or separate out blk and SCSI changes?

 Once we're done merging this patchset, I'll regenerate  repost the 
reqfn reimpl patchset.

 Thanks a lot. :-)
--
tejun
-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-19 Thread Tejun Heo
01_scsi_blk_make_started_requests_ordered.patch

Reordering already started requests is without any real
benefit and causes problems if the request has its
driver-specific resources allocated (as in SCSI).  This patch
makes elv_next_request() set REQ_SOFTBARRIER automatically
when a request is dispatched.

As both as and cfq schedulers don't allow passing requeued
requests, the only behavior change is that requests deferred
by prep_fn won't be passed by other requests.  This change
shouldn't cause any problem.  The only affected driver other
than SCSI is i2o_block.

Signed-off-by: Tejun Heo <[EMAIL PROTECTED]>

 elevator.c |8 
 1 files changed, 4 insertions(+), 4 deletions(-)

Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
08:13:01.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 08:13:33.0 
+0900
@@ -370,11 +370,11 @@ struct request *elv_next_request(request
 
while ((rq = __elv_next_request(q)) != NULL) {
/*
-* just mark as started even if we don't start it, a request
-* that has been delayed should not be passed by new incoming
-* requests
+* just mark as started even if we don't start it.
+* also, as a request that has been delayed should not
+* be passed by new incoming requests, set softbarrier.
 */
-   rq->flags |= REQ_STARTED;
+   rq->flags |= REQ_STARTED | REQ_SOFTBARRIER;
 
if (rq == q->last_merge)
q->last_merge = NULL;

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: [PATCH scsi-misc-2.6 01/05] scsi: make blk layer set REQ_SOFTBARRIER when a request is dispatched

2005-04-19 Thread Tejun Heo
01_scsi_blk_make_started_requests_ordered.patch

Reordering already started requests is without any real
benefit and causes problems if the request has its
driver-specific resources allocated (as in SCSI).  This patch
makes elv_next_request() set REQ_SOFTBARRIER automatically
when a request is dispatched.

As both as and cfq schedulers don't allow passing requeued
requests, the only behavior change is that requests deferred
by prep_fn won't be passed by other requests.  This change
shouldn't cause any problem.  The only affected driver other
than SCSI is i2o_block.

Signed-off-by: Tejun Heo [EMAIL PROTECTED]

 elevator.c |8 
 1 files changed, 4 insertions(+), 4 deletions(-)

Index: scsi-reqfn-export/drivers/block/elevator.c
===
--- scsi-reqfn-export.orig/drivers/block/elevator.c 2005-04-20 
08:13:01.0 +0900
+++ scsi-reqfn-export/drivers/block/elevator.c  2005-04-20 08:13:33.0 
+0900
@@ -370,11 +370,11 @@ struct request *elv_next_request(request
 
while ((rq = __elv_next_request(q)) != NULL) {
/*
-* just mark as started even if we don't start it, a request
-* that has been delayed should not be passed by new incoming
-* requests
+* just mark as started even if we don't start it.
+* also, as a request that has been delayed should not
+* be passed by new incoming requests, set softbarrier.
 */
-   rq-flags |= REQ_STARTED;
+   rq-flags |= REQ_STARTED | REQ_SOFTBARRIER;
 
if (rq == q-last_merge)
q-last_merge = NULL;

-
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/