On 06/03/2014 20:01, Mike Christie wrote:
On 02/27/2014 04:20 PM, Or Gerlitz wrote:
Hi Mike,
We're working now on sort of house cleaning w.r.t the iser transport
code that deals with connections tear-down and resources cleanups.
Examining the iscsi state machine ladder diagram for connection
establishment and login which goes like:
01 transport end-point (create and) connect
02 create session
03 create connection
04 bind iscsi connection to the transport connection
05 set non-negotiated params
06 send/recv login request/response
07 set negotiated params
08 start connection
[etc]
A possible oredering for normal termination could have been something like:
09 stop connection
10 send/recv logout request/response
11 unbind iscsi connection from the transport connection
12 destroy connection
13 destroy session
14 transport end-point disconnect (and destroy)
In reality it's not the case -- there's no unbind primitive and the
actual order is the following:
09 send/recv logout request/response
10 transport end-point disconnect (and destroy)
11 stop connection
12 destroy connection
13 destroy session
Just to make sure, I got it right, correct?
Yeah, no unbind callout for ep.
Now, under this design, we consider the transport end-point disconnect
to be interpreted as
10.1 unbind iscsi connection from the transport connection
10.2 disconnect
Yes. Depending on how the driver's binding from the iscsi connection to
the endpoint connection it can be disconnect then unbind or unbind then
disconnect.
and this design is OK with us. We just want to make sure that after
issuing the disconnect call, the iscsi layer will not attempt to do
anything with the iscsi connection expcet for stopping/destroying it
or binding it to **new** transport end-point which was
created/connected through the recovery flow.
I cannot remember what happened but we did not end up with a unbind
call. I think it was due to iscsi_tcp not needing one since it is done
in userspace. When iser came along and we added those callouts, we just
forgot.
There is a problem with this code because stop_conn is called after
ep_disconnection. IO could still be sent to the driver until the
stop_conn call has completed. I think I remember finding this bug, then
we added the iscsi_suspend_tx call to iscsi_iser_ep_disconnect and some
other drivers, because we could not have users deploy both new userspace
and kernels in some situations.
But yeah, so there is a bug, and as I look at the code again, I am
realizing that for drivers that do not use the iscsi xmit workqueue we
never handled that issue. I can't remember if we just did not have
drivers like that at the time or what.
I am sending this mail now so we can begin to think about the fix and to
make sure you know I am not ignoring this mail :) Currently, I am
kicking around:
1. Add ep/conn unbind callback.
2. Add libiscsi helper which ep_disconnects can call to prevent accesses
to the conn.
For both the LLD, would get what you need, where after ep_disconnect is
called the LLD does not have to worry about conn acceses from libiscsi
except for the ones you mentioned above (stopping, destroy, etc).
Mike,
Thanks for your detailed answer and looking into this. Ariel, myself
and Co from Mellanox are looking into this area too, currently disrupted
by some other issue (starvation of SCSI response processing, we're
probably moving to use the blk iopoll
mechanism for that matter) but plan to get back to this soon.
Or.
In this case, the iser entries planted for the below callbacks would
just be direct pointers into libiscsi service routines
stop connection - iscsi_conn_stop()
destroy connection - iscsi_conn_teardown()
destroy session - new function iscsi_session_destroy (*)
(*) today we use iscsi_session_teardown, and we need to add there
calls to iscsi_host_remove () and iscsi_host_free()
Yeah, if you want to make a new function that does the session and host
destruction, we should probably name it something other than
iscsi_session_destroy to reflect that it does multiple operations.
So how all this sounds? I tend to think that in the past stop_conn
used to be before sending the logout request, is that true?
Maybe. I don't remember.
--
You received this message because you are subscribed to the Google Groups
"open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.
