Mike Christie wrote:
> The nop watch dog is more of generic catch all problems. I think I was
> saying offlist that iscsi_tcp is not able to detect many transport
> problems quickly. For example if you just pulled a cable somewhere in
> the network, iscsi_tcp may not get some event telling us this. If nops
> are off, then we would have to wait for the scsi cmd timer to fire. To
> speed this type of failure up, we do the nops.
> If for iser you can figure out every transport problem with some sort of
> event notification system then you could turn off nops.
understood, both. Here's the thing: nops serve us well, so I don't want to
disable them. We do get event notifications and we can't ignore them.
As it stands the problem I face now is that on one of the error flows, e.g if
you pull the cable and then ep_disconnect is called as of nop out timeout...
it takes the IB CM (Connection Manager) about 1 minute under the params used by
iser to invoke the iser callback which in turn will signal to ep_disconect.
iser doesn't call directly to the IB CM and currently doesn't have control on
the timeout setting of the IB CM and generally speaking this 1 minute value is
good for other possible problems, so at this point I don't think I want to
change that.
Suppose I have a way to make ep_disconnect not to block for more then few milli
seconds, e.g by adding a polling api, the same or similar to ep_poll as you
suggested OR by changing iser to let ep_disconnect return the context to iscsid
and only after (say) a minute at some later point in time
iscsi_destroy_endpoint will be called, am I back in business?
If the answer is YES, keep reading below, but before all what comes next, lets
make sure to clarify this!
--------------------------------------------
> Let me know if you need something like a ep_discocnnect_poll() where the
> ep_disconnect() starts the disconnect, then then new ep_disconnect_poll
> would wait for it to complete.
the poll option is simple and maybe even trivial to implement from iser point
of view but it needs changes in user space and the kernel iscsi transport code,
quite heavy in volume I think.
I started to look on the 2nd option and it gets quite complicated. I don't want
to load you with the iser detailed, but I'd be happy to check with you that its
a valid option. Valid in the sense that its legal for ep_disconnect to return
with the ep not being destroyed.
I looked on commit b40977d95fb3a1898ace6a7d97e4ed1a33a440a4 "iser: fix handling
of scsi cmnds during recovery" and saw the refcounting you added to the iser
end point (struct iser_conn) handling flow. Currently its incremented in one
place (bind) and decremented in either of three places iscsi_iser_conn_{stop,
destroy} and iser_conn_terminate which is called from iser ep disconnect.
iser_conn_terminate blocks on the event of the ib connection being down and
this sometimes takes these 60 seconds. If it legal to really clean the ep after
ep disconnect returns I can try and use this ref count that you added and
increment/decrement it two more times, one for the IB CM and one for QP
buffers. Such that only when the CM delivered the event && all the buffers
where flushed by the HW iser_conn_put will sense the refcount hitting zero and
will call iser_conn_release which will call iscsi_destroy_endpoint. Not that I
have a robust implementation for this nor its free of possible problems... what
do you think which of the two is the way to go or should I try a 3rd method.
Or.
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/open-iscsi?hl=en.
