On 01/23/2015 01:20 AM, Or Gerlitz wrote: > On Fri, Jan 23, 2015 at 12:45 AM, Mike Christie <[email protected] > <mailto:[email protected]>> wrote: > > On 1/22/15, 12:03 PM, Sagi Grimberg wrote: > > On 1/22/2015 3:53 PM, Or Gerlitz wrote: > > On 1/22/2015 1:10 AM, Mike Christie wrote: > > On 01/21/2015 03:14 PM, Or Gerlitz wrote: > > On Wed, Jan 21, 2015 at 8:41 PM, Mike Christie > <[email protected] <mailto:[email protected]> > <mailto:[email protected] > <mailto:[email protected]>>> wrote: > > On 1/21/15, 8:19 AM, Or Gerlitz wrote: > > Hi Mike, > > Long time.. hope you are all well after EOY > holidays and with > energy to > for the MCS debates @ LSF... > > > Hey, > > Hope you are doing well too. > > So back in upstream commit > ea05be3ff043efd44256283d968fa1____bb9a371568 > "iscsi tools: set non negotiated params > early" we are > doing set_param towards the > kernel/transport before sending > the > login > request. > > Running with latest upstream (commit > > 76a441ba0dc0071a19daeac456aa89____8889437efd) we > did dump of all > set_params > done towards iser and I see that Max Recv > DSL and friends are > set after > sending the login, in both discovery and > normal sessions (see > below), is > that a bug? aren't they declarative? > > > Most of them being set are not declarative. > > If we have ImmediateData=yes, but the target > has ImmediateData=No, > then we end up turning off ImmediateData. > MaxBurstLength uses the > min of what each side can support. > > Is the only one currently being set at that > time that is > declarative > MRDSL (TPGT we will not count because we can > sometimes get a > different TGPT than what discovery told us). > The thing about MRDSL > is that we have our own > MaxXmitDataSegmentLength setting. It is > not > a iscsi RFC setting. It is just a initiator > side limit that we > take > into account when setting the final MRDSL. See > usr/login.c:get_op_params_____text_keys(). > > Do you need the values earlier? Is it just > MRSDL? We can pass that > earlier, but then if we find out it is larger than > MaxRecvDataSegmentLength we can update the > value by calling set > param again. We could also just turn that off > for iser if it > causes > problems. > > > What we need in iser is an early (== between bind > and sending login) > set > param call to tell us the MRSDL for the initiator > which is to be > assumed > by the target side. We need not worry if the value > advertized before > the > login nego is larger vs. the max DSL which is going > to be used by the > target. We need all that when running discovery > over iser but if it > makes things simpler, we can have it for Normal > sessions too. Any > chance > you can suggest quick user-space patch or you want > us to dig there... > > It sounds like you just need something like the attached > patch. Let me > know if I am misunderstanding you. > > > > Mike, with your patch the user-space --> transport setting > of MRDSL is > done before sending the login request (but also again after > getting the > login response), see below. > > However, I am not clear why in my setup I get 8192 where > iscsi.conf I > have > > node.conn[0].iscsi.__MaxRecvDataSegmentLength = 262144 > > and later > > discovery.sendtargets.iscsi.__MaxRecvDataSegmentLength = 32768 > > where none of them is 8192 ... is this possible that we send > 32768 in > the SendTargets payload but program 8K to the transport? > > > I think it's because in a discovery session the default (8192) > MRDL is > used. I patched user-space to pass MRDL as non-negotiable and I > get 8192 > as well (just it was before conn_bind). > > > Yes, see other mail. For the userspace code see: > > user/initiator_common.c:iscsi___copy_operational_params() > > > if (session->type == ISCSI_SESSION_TYPE_DISCOVERY) { > /* > * Right now, we only support 8K max for kernel based > * sendtargets discovery, because the recv pdu > buffers are > * limited to this size. > */ > if ((t->caps & CAP_TEXT_NEGO) && > conn->max_recv_dlength > > ISCSI_DEF_MAX_RECV_SEG_LEN) > conn->max_recv_dlength = > ISCSI_DEF_MAX_RECV_SEG_LEN; > > > In the kernel look at libiscsi.c's login_task related code. In > __iscsi_conn_send_pdu you can see we treat the login and text pdus > similarly in that we use the preallocated login_task and its 8K > buffer for both login and text pdus. There is also that check for if > the user tries to do a operation larger than 8K. > > > But I don't think passing it before is necessary. Reusing the login > buffer was our decision. We could have equally allocate a dedicated > buffer at sendtargets text execution time, pre-post it and release > it once we get the response (given we are in a discovery session of > course). > > Thoughts? > > > What problem are you guys trying to solve exactly? If you want to > support larger than 8K buffers then modify the code I mentioned and > the iser code you mention. > > > We want to be able to do send-targets based discovery over iser when the > over-all payload size returned by the target is > 8KB. > > Based on the input I got from Yan that tested that, right now it works > (that is the multiple exchange described @ the spec is implemented right > in iscsid and the target) conditioned that you set MRDSL in iscsid.conf > to be 8K. > > This MRDSL value isn't the factory default and we wanted to remove the > hassle from the users telling them that over iser they need to touch that.
The user does not have to touch it. The iscsi tools will do the right thing for them. > > Again, I don't have the wire dumps in-front of me nor I could get iscsid > to dump that, so I don't know what value was advertised to the target in > the discovery session login request. But, if we have a solution that > rounds down the user-directive for advertised MRDSL > 8KB to be 8KB for > discovery sessions when running over kernel transports - I am OK with that. > It should work ok from the initiator side and most iscsi targets. If the kernel cannot handle large PDUs like with iscsi offload (be2iscsi, cxgb*i and bnx2i) and drivers like iser, then the iscsi tools will break up a large payload into multiple 8K TEXT PDUs. I did that code several years ago, so common targets like netapp, emc, eql, tgt, iet and scst were tested. LIO was not common at the time, so it has probably just got basic testing done. There could be interop issues in corner cases. -- You received this message because you are subscribed to the Google Groups "open-iscsi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/d/optout.
