Re: do discovery through SW transports too
-Original Message- From: Or Gerlitz or.gerl...@gmail.com Reply-To: open-iscsi@googlegroups.com open-iscsi@googlegroups.com Date: Thursday 27 June 2013 2:25 AM To: open-iscsi@googlegroups.com open-iscsi@googlegroups.com Cc: Or Gerlitz ogerl...@mellanox.com, Nicholas A. Bellinger n...@linux-iscsi.org, Alexander Nezhinsky nezhin...@gmail.com Subject: Re: do discovery through SW transports too On Wed, Jun 26, 2013 at 7:29 PM, Mike Christie micha...@cs.wisc.edu wrote: On 06/26/2013 11:25 AM, Mike Christie wrote: Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I Check Linus's tree or James scsi misc branch. I pulled today and see it in both. It is git commit bbcd11aa66ef9ef463434521eb948974ed2a838e Mike, sorry I am not near plain git now, but if I look through the kernel.org git web on Linus tree I see ISCSI_FLASHNODE_DISCOVERY_SESS defined in include/scsi/iscsi_if.h but not ISCSI_PARAM_DISCOVERY_SESS - am I still missing that? its http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/includ e/scsi/iscsi_if.h Patch which added ISCSI_PARAM_DISCOVERY_SESS is not included in any of upstream tree yet. Recently ACKed patch-set includes this parameter. Here is link for same:- http://marc.info/?l=linux-scsim=137226106601859w=2 http://marc.info/?l=linux-scsim=137225028829587w=2 Thanks, Vikas. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 27/06/2013 01:09, Mike Christie wrote:
Mike, sorry I am not near plain git now, but if I look through the
kernel.org git web on Linus tree I see ISCSI_FLASHNODE_DISCOVERY_SESS
defined in include/scsi/iscsi_if.h but not ISCSI_PARAM_DISCOVERY_SESS
- am I still missing that? its
My mistake. I messed up my git trees. Qlogic has been sending me a patch
with it and I have been messing up and merging in different trees
thinking it was already there.
Latest patch was posted the other day:
http://marc.info/?l=linux-scsim=137225028829587w=2
Hi Mike,
Thanks for the clarification, so this isn't yet in James tree, correct?
I just want to make quick test with it, so
1. are there any user space patches?
2. ISCSI_PARAM_DISCOVERY_SESSis bool param, correct?
I was a bit confused from this snippest
+static void qla4xxx_copy_to_sess_conn_
params(struct iscsi_conn *conn,
+struct iscsi_session *sess,
+struct dev_db_entry
*fw_ddb_entry)
+{
+ unsigned long options = 0;
+ uint16_t ddb_link;
+ uint16_t disc_parent;
+
+ options = le16_to_cpu(fw_ddb_entry-options);
+ conn-is_fw_assigned_ipv6 = test_bit(OPT_IS_FW_ASSIGNED_IPV6,
options);
+ sess-auto_snd_tgt_disable = test_bit(OPT_AUTO_SENDTGTS_DISABLE,
+ options);
+ sess-discovery_sess = test_bit(OPT_DISC_SESSION, options);
where is seems that the value is set from below, or this is how things
work for HW-iSCSI cards?
Or.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/27/2013 04:12 AM, Or Gerlitz wrote:
On 27/06/2013 01:09, Mike Christie wrote:
Mike, sorry I am not near plain git now, but if I look through the
kernel.org git web on Linus tree I see ISCSI_FLASHNODE_DISCOVERY_SESS
defined in include/scsi/iscsi_if.h but not ISCSI_PARAM_DISCOVERY_SESS
- am I still missing that? its
My mistake. I messed up my git trees. Qlogic has been sending me a patch
with it and I have been messing up and merging in different trees
thinking it was already there.
Latest patch was posted the other day:
http://marc.info/?l=linux-scsim=137225028829587w=2
Hi Mike,
Thanks for the clarification, so this isn't yet in James tree, correct?
Correct.
I just want to make quick test with it, so
1. are there any user space patches?
No. Qlogic accidentally added the setting but forgot to add userspace
code. They only added userspace code for the flashnode one.
2. ISCSI_PARAM_DISCOVERY_SESSis bool param, correct?
Yeah. 1 for discovery. 0 for not discovery.
I was a bit confused from this snippest
+static void qla4xxx_copy_to_sess_conn_
params(struct iscsi_conn *conn,
+struct iscsi_session *sess,
+struct dev_db_entry
*fw_ddb_entry)
+{
+ unsigned long options = 0;
+ uint16_t ddb_link;
+ uint16_t disc_parent;
+
+ options = le16_to_cpu(fw_ddb_entry-options);
+ conn-is_fw_assigned_ipv6 = test_bit(OPT_IS_FW_ASSIGNED_IPV6,
options);
+ sess-auto_snd_tgt_disable = test_bit(OPT_AUTO_SENDTGTS_DISABLE,
+ options);
+ sess-discovery_sess = test_bit(OPT_DISC_SESSION, options);
where is seems that the value is set from below, or this is how things
work for HW-iSCSI cards?
Basically, yes. They would normally get it from their flash/fw.
Userspace can pass it in too, but it works differently when doing this
flashnode mode support. Do not worry about it.
For iser and all other drivers just do what you asked about before. Do
the normal old set_param stuff before we login.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 26/06/2013 11:15, Nicholas A. Bellinger wrote: Hi Or Co, Ok, sendtargets discovery is now functioning over iser. The updated target patches + v2 changelogs are in for-next commits here: http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=889c8a68b8483a8b3482ac440af3ad7368c58555 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=4ebc2e95f49aca3114acb13b15c3e6f769ee6300 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=58203ef7ae06f17f7ee238491cd7301abe3dfc19 The running initiator + target discovery output is below. Please go ahead and give it a shot on your setup. Nic, using your for-next branch, this works for me too! I am using the below initiator patch and looking on what does it take to have things go correctly. We do need to fix some issues related to negotiation, but I prefer/hope to make discovery work independent of these fixes. index b6d81a8..cec5023 100644 --- a/drivers/infiniband/ulp/iser/iser_initiator.c +++ b/drivers/infiniband/ulp/iser/iser_initiator.c @@ -248,6 +248,8 @@ static int iser_post_rx_bufs(struct iscsi_conn *conn, struct iscsi_hdr *req) WARN_ON(iser_conn-ib_conn-post_recv_buf_count != 1); WARN_ON(atomic_read(iser_conn-ib_conn-post_send_buf_count) != 0); + iser_post_recvl(iser_conn-ib_conn); + iser_dbg(Initially post: %d\n, ISER_MIN_POSTED_RX); /* Initial post receive buffers */ if (iser_post_recvm(iser_conn-ib_conn, ISER_MIN_POSTED_RX)) -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 27/06/2013 14:46, Or Gerlitz wrote: On 26/06/2013 11:15, Nicholas A. Bellinger wrote: Hi Or Co, Ok, sendtargets discovery is now functioning over iser. The updated target patches + v2 changelogs are in for-next commits here: http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=889c8a68b8483a8b3482ac440af3ad7368c58555 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=4ebc2e95f49aca3114acb13b15c3e6f769ee6300 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=58203ef7ae06f17f7ee238491cd7301abe3dfc19 The running initiator + target discovery output is below. Please go ahead and give it a shot on your setup. Nic, using your for-next branch, this works for me too! I am using the below initiator patch and looking on what does it take to have things go correctly. We do need to fix some issues related to negotiation, but I prefer/hope to make discovery work independent of these fixes. index b6d81a8..cec5023 100644 --- a/drivers/infiniband/ulp/iser/iser_initiator.c +++ b/drivers/infiniband/ulp/iser/iser_initiator.c @@ -248,6 +248,8 @@ static int iser_post_rx_bufs(struct iscsi_conn *conn, struct iscsi_hdr *req) WARN_ON(iser_conn-ib_conn-post_recv_buf_count != 1); WARN_ON(atomic_read(iser_conn-ib_conn-post_send_buf_count) != 0); + iser_post_recvl(iser_conn-ib_conn); + iser_dbg(Initially post: %d\n, ISER_MIN_POSTED_RX); /* Initial post receive buffers */ if (iser_post_recvm(iser_conn-ib_conn, ISER_MIN_POSTED_RX)) Things don't shine yet, from time to time LIO crashes, I sent Nic the oops dump in private as I have it with JPG only. Anyway, I think I have the correct iser kernel patch that would allow for this to work once I have the user space patch that does this set param after binding the connection in discovery and before sending the login. Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Sun, 2013-06-23 at 17:47 +0300, Or Gerlitz wrote:
On 23/06/2013 17:40, Or Gerlitz wrote:
there you go, here's the output
isert_cma_handler: event 4 status 0 conn 88011a55d600 id
8801085c5400
RDMA_CM_EVENT_CONNECT_REQUEST:
Entering isert_connect_request cma_id: 8801085c5400, context:
88011a55d600
Using responder_resources: 1 initiator_depth: 4
Set login_buf: 880116a38000 login_req_buf: 880116a38000
login_rsp_buf: 880116a3a000
Using 3 CQs, device mlx4_0 supports 3 vectors
[...]
and here's the initiator output (if you set debug_level=3 for ib_iser)
iser: iser_rcv_completion:op 0x3f itt 0x dlen 0
iser: iscsi_iser_recv:wrong datalen 48 (hdr), 0 (IB)
This means that LIO sent ISCSI_OP_REJECT (0x3f) and that there's a bug
for reject since the iSCSI header
says there are 48 bytes beyond the BHS but on the wire [1] there where none
Hi Or Co,
Ok, sendtargets discovery is now functioning over iser. The updated
target patches + v2 changelogs are in for-next commits here:
http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=889c8a68b8483a8b3482ac440af3ad7368c58555
http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=4ebc2e95f49aca3114acb13b15c3e6f769ee6300
http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=58203ef7ae06f17f7ee238491cd7301abe3dfc19
The running initiator + target discovery output is below. Please go
ahead and give it a shot on your setup.
One thing to note, this currently *only* works with open-iscsi when the
TEXT_RSP payload posted by the target is = 128 bytes (eg: equal to or
less than the initiator posted ISER_RECV_DATA_SEG_LEN), otherwise a
IB_WC_LOC_LEN_ERR error will be generated by the initiator..
So that said, I think we need to go ahead and bump the open-iscsi side
ISER_RECV_DATA_SEG_LEN constant to 8192 in order to match what is
actually advertised by the initiator's MaxRecvDataSegmentLength during
discovery negotiation.
Or and Mike, here is a quick patch on the initiator side.. Ideally this
would follow the negotiated MRDSL, but it's certainly better than
hitting IB_WC_LOC_LEN_ERR errors during (most) typical sendtargets
cases. WDYT..?
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.h
b/drivers/infiniband/ulp/iser/iscsi_iser.h
index 4f069c0..2fab32c 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.h
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.h
@@ -153,7 +153,7 @@ struct iser_cm_hdr {
/* Constant PDU lengths calculations */
#define ISER_HEADERS_LEN (sizeof(struct iser_hdr) + sizeof(struct iscsi_hdr))
-#define ISER_RECV_DATA_SEG_LEN 128
+#define ISER_RECV_DATA_SEG_LEN 8192
#define ISER_RX_PAYLOAD_SIZE (ISER_HEADERS_LEN + ISER_RECV_DATA_SEG_LEN)
#define ISER_RX_LOGIN_SIZE (ISER_HEADERS_LEN + ISCSI_DEF_MAX_RECV_SEG_LEN)
Also, as you reported above there is still a problem somewhere with the
posting/handling of REJECTs PDUs. I'm currently tracking this down, and
is a bug separate from the v3.11 changes.
More on that soon..
--nab
Initiator side:
[ 251.926481] iscsi: registered transport (iser)
[ 251.926705] caps for iscsi transport: 0x0089
[ 296.946755] iser: iser_connect:connecting to: 10.100.0.1, port 0xbd0c
[ 296.951793] iser: iscsi_iser_ep_poll:ib conn 8802354699b8 rc = 0
[ 296.952319] iser: iser_cma_handler:event 0 status 0 conn 8802354699b8 id
880234325c00
[ 296.952737] iser: iser_create_device_ib_res:using 4 CQs, device mlx4_0
supports 19 vectors
[ 296.968061] iser: iser_cma_handler:event 2 status 0 conn 8802354699b8 id
880234325c00
[ 296.978296] iser: iser_create_ib_conn_res:cq index 0 used for ib_conn
8802354699b8
[ 296.979763] iser: iser_create_ib_conn_res:setting conn 8802354699b8
cma_id 880234325c00: fmr_pool 880236a23640 qp 8802346c6e00
[ 297.011751] iser: iser_cma_handler:event 9 status 0 conn 8802354699b8 id
880234325c00
[ 297.954147] iser: iscsi_iser_ep_poll:ib conn 8802354699b8 rc = 1
[ 297.954614] scsi6 : iSCSI Initiator over iSER, v.0.1
[ 297.955350] iser: iscsi_iser_conn_bind:binding iscsi/iser conn
88022b7fadf0 88022b7fafd0 to ib_conn 8802354699b8
[ 297.955807] iser: iscsi_iser_mtask_xmit:mtask xmit [cid 0 itt 0x0]
[ 297.955810] iser: iser_post_rx_bufs:req op 43 flags 87
[ 297.955811] iser: iser_post_rx_bufs:Initially post: 32
[ 297.966140] iser: iser_rcv_completion:op 0x23 itt 0x0 dlen 176
[ 297.966143] iser: iscsi_iser_recv:aligned datalen (173) hdr, 176 (IB)
[ 297.966152] iser: iser_cq_tasklet_fn:got 1 rx 1 tx completions
[ 297.966576] iser: iscsi_iser_mtask_xmit:mtask xmit [cid 0 itt 0x0]
[ 297.966581] iser: iser_post_rx_bufs:req op 4 flags 80
[ 297.982123] iser: iser_rcv_completion:op 0x24 itt 0x0 dlen 84
[ 297.982126] iser: iscsi_iser_recv:aligned datalen (83) hdr, 84 (IB)
[ 297.982135] iser: iser_cq_tasklet_fn:got 1 rx 1 tx completions
[
Re: do discovery through SW transports too
On 26/06/2013 11:15, Nicholas A. Bellinger wrote: Hi Or Co, Ok, sendtargets discovery is now functioning over iser. forgot to say: cool! thanks for the hard work get that done. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 20/06/2013 19:28, Mike Christie wrote: On 06/20/2013 11:10 AM, Or Gerlitz wrote: On 19/06/2013 00:47, Mike Christie wrote: On 06/18/2013 10:35 AM, Or Gerlitz wrote: Now, back to the initiator, Mike, is there a chance that the initiator SENDS the key TargetRecvDataSegmentLength? this shouldn't be the case according to the iser IETF spec. See add_params_transport_specific() in login.c. Add some prints in there to see what is set at that time and getting sent. Hi Mike, I see that in user space, session has type field, does this propagate into the kernel? in other words, can the iser kernel module somehow No. realize if the session being created is normal or discovery?if not, is there an easy way to add that? Pass it down like any other param. There is a param ISCSI_PARAM_DISCOVERY_SESS for this, but I do not think userspace is sending it down yet. Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I assume I can add it on both ends. Now, according to the ladder diagram http://www.open-iscsi.org/docs/open-iscsi-1.jpg set_param is called after the login response was received to provide the kernel transport driver with the negotiated params. Are you OK to add a set_param (with the session type) call after binding the transport connection but before sending the login request to the target? Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 26/06/2013 11:15, Nicholas A. Bellinger wrote: On Sun, 2013-06-23 at 17:47 +0300, Or Gerlitz wrote: On 23/06/2013 17:40, Or Gerlitz wrote: there you go, here's the output isert_cma_handler: event 4 status 0 conn 88011a55d600 id 8801085c5400 RDMA_CM_EVENT_CONNECT_REQUEST: Entering isert_connect_request cma_id: 8801085c5400, context: 88011a55d600 Using responder_resources: 1 initiator_depth: 4 Set login_buf: 880116a38000 login_req_buf: 880116a38000 login_rsp_buf: 880116a3a000 Using 3 CQs, device mlx4_0 supports 3 vectors [...] and here's the initiator output (if you set debug_level=3 for ib_iser) iser: iser_rcv_completion:op 0x3f itt 0x dlen 0 iser: iscsi_iser_recv:wrong datalen 48 (hdr), 0 (IB) This means that LIO sent ISCSI_OP_REJECT (0x3f) and that there's a bug for reject since the iSCSI header says there are 48 bytes beyond the BHS but on the wire [1] there where none Hi Or Co, Ok, sendtargets discovery is now functioning over iser. The updated target patches + v2 changelogs are in for-next commits here: http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=889c8a68b8483a8b3482ac440af3ad7368c58555 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=4ebc2e95f49aca3114acb13b15c3e6f769ee6300 http://git.kernel.org/cgit/linux/kernel/git/nab/target-pending.git/commit/?h=for-nextid=58203ef7ae06f17f7ee238491cd7301abe3dfc19 The running initiator + target discovery output is below. Please go ahead and give it a shot on your setup. One thing to note, this currently *only* works with open-iscsi when the TEXT_RSP payload posted by the target is = 128 bytes (eg: equal to or less than the initiator posted ISER_RECV_DATA_SEG_LEN), otherwise a IB_WC_LOC_LEN_ERR error will be generated by the initiator.. yes, I have started to look on that, checking few options how to solve, will let you know soon. Or -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/26/2013 10:47 AM, Or Gerlitz wrote: On 20/06/2013 19:28, Mike Christie wrote: On 06/20/2013 11:10 AM, Or Gerlitz wrote: On 19/06/2013 00:47, Mike Christie wrote: On 06/18/2013 10:35 AM, Or Gerlitz wrote: Now, back to the initiator, Mike, is there a chance that the initiator SENDS the key TargetRecvDataSegmentLength? this shouldn't be the case according to the iser IETF spec. See add_params_transport_specific() in login.c. Add some prints in there to see what is set at that time and getting sent. Hi Mike, I see that in user space, session has type field, does this propagate into the kernel? in other words, can the iser kernel module somehow No. realize if the session being created is normal or discovery?if not, is there an easy way to add that? Pass it down like any other param. There is a param ISCSI_PARAM_DISCOVERY_SESS for this, but I do not think userspace is sending it down yet. Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I Check Linus's tree or James scsi misc branch. I pulled today and see it in both. assume I can add it on both ends. Now, according to the ladder diagram It just needs support in open-iscsi/usr. The open-iscsi/kernel code should not be touched. It is only for old kernels. http://www.open-iscsi.org/docs/open-iscsi-1.jpg set_param is called after the login response was received to provide the kernel transport driver with the negotiated params. Are you OK to add a set_param (with the session type) call after binding the transport connection but before sending the login request to the target? It is ok to call set_param at this time instead of just after login. We do this for network settings for some offload cards already. The diagram on that web site is old. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/26/2013 11:25 AM, Mike Christie wrote: Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I Check Linus's tree or James scsi misc branch. I pulled today and see it in both. It is git commit bbcd11aa66ef9ef463434521eb948974ed2a838e -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Wed, Jun 26, 2013 at 7:29 PM, Mike Christie micha...@cs.wisc.edu wrote: On 06/26/2013 11:25 AM, Mike Christie wrote: Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I Check Linus's tree or James scsi misc branch. I pulled today and see it in both. It is git commit bbcd11aa66ef9ef463434521eb948974ed2a838e Mike, sorry I am not near plain git now, but if I look through the kernel.org git web on Linus tree I see ISCSI_FLASHNODE_DISCOVERY_SESS defined in include/scsi/iscsi_if.h but not ISCSI_PARAM_DISCOVERY_SESS - am I still missing that? its http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/scsi/iscsi_if.h -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/26/2013 03:55 PM, Or Gerlitz wrote: On Wed, Jun 26, 2013 at 7:29 PM, Mike Christie micha...@cs.wisc.edu wrote: On 06/26/2013 11:25 AM, Mike Christie wrote: Mike, I don't see a sign for ISCSI_PARAM_DISCOVERY_SESS in the upstream open-iscsi user or kernel code nor in the upstream kernel... but I Check Linus's tree or James scsi misc branch. I pulled today and see it in both. It is git commit bbcd11aa66ef9ef463434521eb948974ed2a838e Mike, sorry I am not near plain git now, but if I look through the kernel.org git web on Linus tree I see ISCSI_FLASHNODE_DISCOVERY_SESS defined in include/scsi/iscsi_if.h but not ISCSI_PARAM_DISCOVERY_SESS - am I still missing that? its My mistake. I messed up my git trees. Qlogic has been sending me a patch with it and I have been messing up and merging in different trees thinking it was already there. Latest patch was posted the other day: http://marc.info/?l=linux-scsim=137225028829587w=2 http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/scsi/iscsi_if.h Yeah, that is basically the same setting but for flashnode (where iscsi hbas do this stuff in firmware and flash) support. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 20/06/2013 23:31, Nicholas A. Bellinger wrote: On Thu, 2013-06-20 at 19:01 +0300, Or Gerlitz wrote: On 19/06/2013 04:32, Nicholas A. Bellinger wrote: So I'm pretty sure this is due to iscsi_target_parameters.c: iscsi_set_keys_irrelevant_for_discovery() currently clearing INITIATORRECVDATASEGMENTLENGTH and TARGETRECVDATASEGMENTLENGTH for all discovery scenarios.. As I'm still not yet able to force iser discovery to occur on the initiator side, can you give the following patch a shot on your setup..? --nab Nic, I see that this patch exists in the queue branch of your target-pending tree, so I tested with that branch and still faced problems. So what happens now is that the login of the discovery session works fine! nod, thanks for verifying that particular bit.. But, the sending over the IB connection of the TEXT PDU containing Sendtargets=Allwhich is 16 byte long iscsiadm: in kstart_conn iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: sending text pdu with CmdSN 1, exp_statsn 1 iscsiadm: SendTargets=All iscsiadm: in ksend_pdu_begin iscsiadm: send PDU began for hdr 48 bytes and data 16 bytes iscsiadm: in kwritev iscsiadm: wrote 48 bytes of PDU header iscsiadm: in kwritev iscsiadm: wrote 16 bytes of PDU data iscsiadm: in ksend_pdu_end iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: send PDU finished for conn 28:0 fails, that is the kernel iser driver gets TX completion with error IB_WC_REM_OP_ERR iser: iser_drain_tx_cq:tx id 88006facac98 status 11 vend_err 89 Is there a chance you don't post RX buffer which is large enough to hold these 48 + 16 bytes in case this is discovery and not normal session? AFAICT, the isert_put_login_tx() - isert_alloc_rx_descriptors() - isert_post_recv() is happening for both types of sessions, before the last login response is posted and login state moves to full feature phase. Can you post a target side dump with dynamic debugging enabled with the following..? echo 'module iscsi_target_mod +p' /debug/dynamic_debug/control echo 'module ib_isert +p' /debug/dynamic_debug/control --nab there you go, here's the output isert_cma_handler: event 4 status 0 conn 88011a55d600 id 8801085c5400 RDMA_CM_EVENT_CONNECT_REQUEST: Entering isert_connect_request cma_id: 8801085c5400, context: 88011a55d600 Using responder_resources: 1 initiator_depth: 4 Set login_buf: 880116a38000 login_req_buf: 880116a38000 login_rsp_buf: 880116a3a000 Using 3 CQs, device mlx4_0 supports 3 vectors devattr-max_sge: 32 devattr-max_sge_rd: 0 isert_conn_setup_qp: Using min_index: 0 isert_conn_setup_qp cma_id-device: 8801168b isert_conn_setup_qp conn_pd-device: 8801168b rdma_create_qp() returned success . isert_connect_request() waking up np_accept_wq: 88011a55d600 Setup sge: addr: 116a38000 length: 8268 0x00017200 ib_post_recv(): returned success Before rdma_accept . After rdma_accept . Processing isert_accept_np: isert_conn: 880111939800 Added timeout timer to iSCSI login request for 15 seconds. Moving to TARG_CONN_STATE_XPT_UP. isert_get_login_rx before conn_login_comp conn: 88011986 isert_cma_handler: event 9 status 0 conn 880111939800 id 8801085c5400 RDMA_CM_EVENT_ESTABLISHED ISER login_buf: Using rx_dma: 0x116a38000, rx_buflen: 8268 iSCSI opcode: 0x43, ITT: 0x, flags: 0x87 dlen: 248 Using login payload size: 248, rx_buflen: 248 MAX_KEY_VALUE_PAIRS: 8192 iSERT: Decremented post_recv_buf_count: 0 isert_get_login_rx processing login-req: 880106de1024 Received iSCSI login request from 192.168.20.201 on IB/iSER Network Portal 192.168.20.199:3271 Moving to TARG_CONN_STATE_IN_LOGIN. i_buf: iqn.1994-05.com.redhat:308a2565a30, s_buf: Discovery, t_buf: (null) Got key: InitiatorName=iqn.1994-05.com.redhat:308a2565a30 iSCSI Parameter updated to InitiatorName=iqn.1994-05.com.redhat:308a2565a30 Got key: InitiatorAlias=xena017-4 iSCSI Parameter updated to InitiatorAlias=xena017-4 Got key: SessionType=Discovery iSCSI Parameter updated to SessionType=Discovery Got key: HeaderDigest=None iSCSI Parameter updated to HeaderDigest=None Got key: DataDigest=None iSCSI Parameter updated to DataDigest=None Got key: DefaultTime2Wait=2 iSCSI Parameter updated to DefaultTime2Wait=2 Got key: DefaultTime2Retain=0 iSCSI Parameter updated to DefaultTime2Retain=0 Got key: IFMarker=No iSCSI Parameter updated to IFMarker=No Got key: OFMarker=No iSCSI Parameter updated to OFMarker=No Got key: ErrorRecoveryLevel=0 iSCSI Parameter updated to ErrorRecoveryLevel=0 Got key: MaxRecvDataSegmentLength=8192 iSCSI Parameter updated to MaxRecvDataSegmentLength=8192 Saving op-MaxRecvDataSegmentLength from original initiator received value: 8192 iSCSI Parameter updated to MaxRecvDataSegmentLength=262144 Updated MaxRecvDataSegmentLength to target MXDSL value: 262144 iSCSI Parameter updated to IFMarker=No
Re: do discovery through SW transports too
On 23/06/2013 17:40, Or Gerlitz wrote: there you go, here's the output isert_cma_handler: event 4 status 0 conn 88011a55d600 id 8801085c5400 RDMA_CM_EVENT_CONNECT_REQUEST: Entering isert_connect_request cma_id: 8801085c5400, context: 88011a55d600 Using responder_resources: 1 initiator_depth: 4 Set login_buf: 880116a38000 login_req_buf: 880116a38000 login_rsp_buf: 880116a3a000 Using 3 CQs, device mlx4_0 supports 3 vectors [...] and here's the initiator output (if you set debug_level=3 for ib_iser) iser: iser_rcv_completion:op 0x3f itt 0x dlen 0 iser: iscsi_iser_recv:wrong datalen 48 (hdr), 0 (IB) This means that LIO sent ISCSI_OP_REJECT (0x3f) and that there's a bug for reject since the iSCSI header says there are 48 bytes beyond the BHS but on the wire [1] there where none Or. [1] its ib_wc-byte_len minus the iscsi/iser headers, see this code in the initiator iscsi_iser_recv(conn-iscsi_conn, hdr, rx_desc-data, rx_xfer_len - ISER_HEADERS_LEN); # dmesg iser: iser_connect:connecting to: 192.168.20.199, port 0xc70c iser: iser_cma_handler:event 0 status 0 conn 880070ee33f0 id 880070916800 iser: iscsi_iser_ep_poll:ib conn 880070ee33f0 rc = 0 iser: iser_cma_handler:event 2 status 0 conn 880070ee33f0 id 880070916800 fmr_pool: Device mlx4_0 does not support FMRs iser: iser_create_ib_conn_res:FMRs are not supported, using unaligned mode iser: iser_create_ib_conn_res:cq index 1 used for ib_conn 880070ee33f0 iser: iser_create_ib_conn_res:setting conn 880070ee33f0 cma_id 880070916800: fmr_pool (null) qp 880070916400 iser: iser_cma_handler:event 9 status 0 conn 880070ee33f0 id 880070916800 iser: iscsi_iser_ep_poll:ib conn 880070ee33f0 rc = 1 scsi2 : iSCSI Initiator over iSER iser: iscsi_iser_conn_bind:binding iscsi/iser conn 88007a740490 88007a740700 to ib_conn 880070ee33f0 iser: iscsi_iser_mtask_xmit:mtask xmit [cid 0 itt 0x0] iser: iser_post_rx_bufs:req op 43 flags 87 iser: iser_post_rx_bufs:Initially post: 32 iser: iser_rcv_completion:op 0x23 itt 0x0 dlen 176 iser: iscsi_iser_recv:aligned datalen (173) hdr, 176 (IB) iser: iser_cq_tasklet_fn:got 1 rx 1 tx completions iser: iscsi_iser_mtask_xmit:mtask xmit [cid 0 itt 0x0] iser: iser_post_rx_bufs:req op 4 flags 80 iser: iser_rcv_completion:op 0x3f itt 0x dlen 0 iser: iscsi_iser_recv:wrong datalen 48 (hdr), 0 (IB) connection3:0: detected conn error (1006) iser: iser_cq_tasklet_fn:got 1 rx 1 tx completions iser: iscsi_iser_ep_disconnect:ib conn 880070ee33f0 state 2 iser: iser_cq_tasklet_fn:got 64 rx 0 tx completions -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Thu, Jun 20, 2013 at 11:31 PM, Nicholas A. Bellinger n...@linux-iscsi.org wrote: On Thu, 2013-06-20 at 19:01 +0300, Or Gerlitz wrote: On 19/06/2013 04:32, Nicholas A. Bellinger wrote: So I'm pretty sure this is due to iscsi_target_parameters.c: iscsi_set_keys_irrelevant_for_discovery() currently clearing INITIATORRECVDATASEGMENTLENGTH and TARGETRECVDATASEGMENTLENGTH for all discovery scenarios.. As I'm still not yet able to force iser discovery to occur on the initiator side, can you give the following patch a shot on your setup..? --nab Nic, I see that this patch exists in the queue branch of your target-pending tree, so I tested with that branch and still faced problems. So what happens now is that the login of the discovery session works fine! nod, thanks for verifying that particular bit.. But, the sending over the IB connection of the TEXT PDU containing Sendtargets=Allwhich is 16 byte long iscsiadm: in kstart_conn iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: sending text pdu with CmdSN 1, exp_statsn 1 iscsiadm: SendTargets=All iscsiadm: in ksend_pdu_begin iscsiadm: send PDU began for hdr 48 bytes and data 16 bytes iscsiadm: in kwritev iscsiadm: wrote 48 bytes of PDU header iscsiadm: in kwritev iscsiadm: wrote 16 bytes of PDU data iscsiadm: in ksend_pdu_end iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: send PDU finished for conn 28:0 fails, that is the kernel iser driver gets TX completion with error IB_WC_REM_OP_ERR iser: iser_drain_tx_cq:tx id 88006facac98 status 11 vend_err 89 Is there a chance you don't post RX buffer which is large enough to hold these 48 + 16 bytes in case this is discovery and not normal session? AFAICT, the isert_put_login_tx() - isert_alloc_rx_descriptors() - isert_post_recv() is happening for both types of sessions, before the last login response is posted and login state moves to full feature phase. Can you post a target side dump with dynamic debugging enabled with the following..? echo 'module iscsi_target_mod +p' /debug/dynamic_debug/control echo 'module ib_isert +p' /debug/dynamic_debug/control I did that yesterday but there were no prints in the dmesg, should they be in the dmesg or I should see that by $ cat /debug/dynamic_debug/SOMETHING -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 19/06/2013 04:32, Nicholas A. Bellinger wrote: So I'm pretty sure this is due to iscsi_target_parameters.c: iscsi_set_keys_irrelevant_for_discovery() currently clearing INITIATORRECVDATASEGMENTLENGTH and TARGETRECVDATASEGMENTLENGTH for all discovery scenarios.. As I'm still not yet able to force iser discovery to occur on the initiator side, can you give the following patch a shot on your setup..? --nab Nic, I see that this patch exists in the queue branch of your target-pending tree, so I tested with that branch and still faced problems. So what happens now is that the login of the discovery session works fine! But, the sending over the IB connection of the TEXT PDU containing Sendtargets=Allwhich is 16 byte long iscsiadm: in kstart_conn iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: sending text pdu with CmdSN 1, exp_statsn 1 iscsiadm: SendTargets=All iscsiadm: in ksend_pdu_begin iscsiadm: send PDU began for hdr 48 bytes and data 16 bytes iscsiadm: in kwritev iscsiadm: wrote 48 bytes of PDU header iscsiadm: in kwritev iscsiadm: wrote 16 bytes of PDU data iscsiadm: in ksend_pdu_end iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: send PDU finished for conn 28:0 fails, that is the kernel iser driver gets TX completion with error IB_WC_REM_OP_ERR iser: iser_drain_tx_cq:tx id 88006facac98 status 11 vend_err 89 Is there a chance you don't post RX buffer which is large enough to hold these 48 + 16 bytes in case this is discovery and not normal session? -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/20/2013 11:10 AM, Or Gerlitz wrote:
On 19/06/2013 00:47, Mike Christie wrote:
On 06/18/2013 10:35 AM, Or Gerlitz wrote:
Now, back to the initiator, Mike, is there a chance that the initiator
SENDS the key TargetRecvDataSegmentLength? this shouldn't be the case
according to the iser IETF spec.
See add_params_transport_specific() in login.c. Add some prints in there
to see what is set at that time and getting sent.
Hi Mike,
I see that in user space, session has type field, does this propagate
into the kernel? in other words, can the iser kernel module somehow
No.
realize if the session being created is normal or discovery?if not, is
there an easy way to add that?
Pass it down like any other param. There is a param
ISCSI_PARAM_DISCOVERY_SESS for this, but I do not think userspace is
sending it down yet.
Looking on add_params_transport_specific, I see that it checks the
session type, but its only
called when the type is NORMAL... also, is there any reason for the
You lost me. So it is not sent during discovery like we want right? Did
you say that want it sent because it will then hit the double bug where
it ends up working.
initiator to send the
TargetRecvDataSegmentLength key?
No idea off the top of my head :) You guys added that code for iser right?
Or.
tatic int
add_params_transport_specific(iscsi_session_t *session, int cid,
struct iscsi_hdr *pdu, char *data,
int max_data_length)
{
char value[AUTH_STR_MAX_LEN];
iscsi_conn_t *conn = session-conn[cid];
if (session-type == ISCSI_SESSION_TYPE_DISCOVERY ||
!session-t-template-rdma) {
sprintf(value, %d, conn-max_recv_dlength);
if (!iscsi_add_text(pdu, data, max_data_length,
MaxRecvDataSegmentLength, value))
return 0;
} else {
sprintf(value, %d, conn-max_recv_dlength);
if (!iscsi_add_text(pdu, data, max_data_length,
InitiatorRecvDataSegmentLength,
value))
return 0;
sprintf(value, %d, conn-max_xmit_dlength);
if (!iscsi_add_text(pdu, data, max_data_length,
TargetRecvDataSegmentLength,
value))
return 0;
if (!iscsi_add_text(pdu, data, max_data_length,
RDMAExtensions, Yes))
return 0;
}
return 1;
}
Or.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Thu, 2013-06-20 at 19:01 +0300, Or Gerlitz wrote: On 19/06/2013 04:32, Nicholas A. Bellinger wrote: So I'm pretty sure this is due to iscsi_target_parameters.c: iscsi_set_keys_irrelevant_for_discovery() currently clearing INITIATORRECVDATASEGMENTLENGTH and TARGETRECVDATASEGMENTLENGTH for all discovery scenarios.. As I'm still not yet able to force iser discovery to occur on the initiator side, can you give the following patch a shot on your setup..? --nab Nic, I see that this patch exists in the queue branch of your target-pending tree, so I tested with that branch and still faced problems. So what happens now is that the login of the discovery session works fine! nod, thanks for verifying that particular bit.. But, the sending over the IB connection of the TEXT PDU containing Sendtargets=Allwhich is 16 byte long iscsiadm: in kstart_conn iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: sending text pdu with CmdSN 1, exp_statsn 1 iscsiadm: SendTargets=All iscsiadm: in ksend_pdu_begin iscsiadm: send PDU began for hdr 48 bytes and data 16 bytes iscsiadm: in kwritev iscsiadm: wrote 48 bytes of PDU header iscsiadm: in kwritev iscsiadm: wrote 16 bytes of PDU data iscsiadm: in ksend_pdu_end iscsiadm: in __kipc_call iscsiadm: in kwritev iscsiadm: in nlpayload_read iscsiadm: in nlpayload_read iscsiadm: send PDU finished for conn 28:0 fails, that is the kernel iser driver gets TX completion with error IB_WC_REM_OP_ERR iser: iser_drain_tx_cq:tx id 88006facac98 status 11 vend_err 89 Is there a chance you don't post RX buffer which is large enough to hold these 48 + 16 bytes in case this is discovery and not normal session? AFAICT, the isert_put_login_tx() - isert_alloc_rx_descriptors() - isert_post_recv() is happening for both types of sessions, before the last login response is posted and login state moves to full feature phase. Can you post a target side dump with dynamic debugging enabled with the following..? echo 'module iscsi_target_mod +p' /debug/dynamic_debug/control echo 'module ib_isert +p' /debug/dynamic_debug/control --nab -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 15/06/2013 11:25, Nicholas A. Bellinger wrote: Hi Or Mike, On Tue, 2013-06-11 at 08:04 +0300, Or Gerlitz wrote: On 06/06/2013 18:01, Mike Christie wrote: However, above I am not talking about that or doing discovery over a normal session in general. I was just trying to get clarification for what you wanted. I was not sure if there was some new iser spec stuff that I missed and you wanted to implement. Hi Mike, Its not new spec stuff, its just environments where plain TCP services might be out of hand for either of the sides and we don't want that to be an obstacle to deploy iser. No luck yet generating discovery ib_iser session traffic with mnc's initial patch + latest open-iscsi.git userspace. iscsiadm is currently defaulting to tcp for all sendtargets discovery: root@barret:# /sbin/iscsiadm -m discovery -t sendtargets --portal 10.100.0.1:3261 -I iser -d 4 iscsiadm: Max file limits 1024 1024 iscsiadm: Matched transport iser iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/handle' with attribute value '18446744072102379920' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/caps' with attribute value '0x9' iscsiadm: Matched transport tcp iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/handle' with attribute value '18446744072101696624' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/caps' with attribute value '0x39' iscsiadm: Could not match iface[hw=,ip=,net_if=,iscsi_if=iser] to host. iscsiadm: starting sendtargets discovery, address 10.100.0.1:3261, iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/tcp/caps' iscsiadm: iscsi_create_session discovery ep connect iser, caps: 0x0009 iscsiadm: connecting to 10.100.0.1:3261 iscsiadm: connected local port 60622 to 10.100.0.1:3261 iscsiadm: connected to discovery address 10.100.0.1 iscsiadm: login response status iscsiadm: discovery process to 10.100.0.1:3261 exiting iscsiadm: disconnecting conn 0x6d2f40, fd 3 192.168.1.69:3260,1 iqn.2003-01.org.linux-iscsi.tifa.x8664:sn.62d7638ae251 10.100.0.1:3260,1 iqn.foo 10.100.0.1:3261,1 iqn.foo 192.168.1.69:3260,1 iqn.foo 10.100.0.1:3260,1 iqn.foo 10.100.0.1:3261,1 iqn.foo 10.200.0.1:3260,1 iqn.foo2 10.200.0.1:3261,1 iqn.foo2 192.168.1.69:3260,1 iqn.foo2 10.200.0.1:3260,1 iqn.foo2 10.200.0.1:3261,1 iqn.foo2 Following the debug pointer, CAP_TEXT_NEGO appears to be missing from session-t-caps during iscsi_create_session() setup usage: iscsiadm: iscsi_create_session discovery ep connect iser, caps: 0x0009 Any reason why userspace would not matching iscsi_iser_transport-caps w/ CAP_TEXT_NEGO set (0x0089)..? Hi Nic, Mike in my environment (user space running latest open-iscsi tools, kernel in both initiator and target based on your queue branch (3.10-rc5) + Nic'sLIO patches to enable TEXT / Sendtargets support by iser + the initiator patch that advertizes CAP_TEXT_NEGO --the user space initiator tools do see the text cap and attempt to issue the discovery session through iser, but this somehow fails with LIO, with the following print which I have already reported earlier: xena017-3 kernel: i_buf: iqn.1994-05.com.redhat:308a2565a30, s_buf: Discovery, t_buf: (null) xena017-3 kernel: Unable to accept text parameter length: 16greater than MaxXmitDataSegmentLength 0. Looking in
Re: do discovery through SW transports too
On 06/18/2013 10:35 AM, Or Gerlitz wrote: Now, back to the initiator, Mike, is there a chance that the initiator SENDS the key TargetRecvDataSegmentLength? this shouldn't be the case according to the iser IETF spec. See add_params_transport_specific() in login.c. Add some prints in there to see what is set at that time and getting sent. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Tue, 2013-06-18 at 18:35 +0300, Or Gerlitz wrote:
On 15/06/2013 11:25, Nicholas A. Bellinger wrote:
Hi Or Mike,
On Tue, 2013-06-11 at 08:04 +0300, Or Gerlitz wrote:
On 06/06/2013 18:01, Mike Christie wrote:
However, above I am not talking about that or doing discovery over a
normal session in general. I was just trying to get clarification for
what you wanted. I was not sure if there was some new iser spec stuff
that I missed and you wanted to implement.
Hi Mike,
Its not new spec stuff, its just environments where plain TCP services
might be out of hand for either of the sides and we don't want that to
be an obstacle to deploy iser.
No luck yet generating discovery ib_iser session traffic with mnc's
initial patch + latest open-iscsi.git userspace.
iscsiadm is currently defaulting to tcp for all sendtargets discovery:
root@barret:# /sbin/iscsiadm -m discovery -t sendtargets --portal
10.100.0.1:3261 -I iser -d 4
SNIP
Following the debug pointer, CAP_TEXT_NEGO appears to be missing from
session-t-caps during iscsi_create_session() setup usage:
iscsiadm: iscsi_create_session discovery ep connect iser, caps: 0x0009
Any reason why userspace would not matching iscsi_iser_transport-caps
w/ CAP_TEXT_NEGO set (0x0089)..?
Hi Nic, Mike
in my environment (user space running latest open-iscsi tools, kernel in
both initiator and target based on your queue branch (3.10-rc5) +
Nic'sLIO patches to enable TEXT / Sendtargets support by iser + the
initiator patch that advertizes CAP_TEXT_NEGO --the user space
initiator tools do see the text cap and attempt to issue the discovery
session through iser, but this somehow fails with LIO, with the
following print which I have already reported earlier:
xena017-3 kernel: i_buf: iqn.1994-05.com.redhat:308a2565a30, s_buf:
Discovery, t_buf: (null)
xena017-3 kernel: Unable to accept text parameter length: 16greater than
MaxXmitDataSegmentLength 0.
So I'm pretty sure this is due to iscsi_target_parameters.c:
iscsi_set_keys_irrelevant_for_discovery() currently clearing
INITIATORRECVDATASEGMENTLENGTH and TARGETRECVDATASEGMENTLENGTH for
all discovery scenarios..
As I'm still not yet able to force iser discovery to occur on the
initiator side, can you give the following patch a shot on your setup..?
--nab
diff --git a/drivers/target/iscsi/iscsi_target_login.c
b/drivers/target/iscsi/iscsi_target_login.c
index bb5d5c5..f42eb84 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -369,7 +369,7 @@ static int iscsi_login_zero_tsih_s2(
if (sess-sess_ops-SessionType)
return iscsi_set_keys_irrelevant_for_discovery(
- conn-param_list);
+ conn-param_list, iser);
na = iscsit_tpg_get_node_attrib(sess);
diff --git a/drivers/target/iscsi/iscsi_target_parameters.c
b/drivers/target/iscsi/iscsi_target_para
index e382221..0b8c819 100644
--- a/drivers/target/iscsi/iscsi_target_parameters.c
+++ b/drivers/target/iscsi/iscsi_target_parameters.c
@@ -545,7 +545,8 @@ int iscsi_set_keys_to_negotiate(
}
int iscsi_set_keys_irrelevant_for_discovery(
- struct iscsi_param_list *param_list)
+ struct iscsi_param_list *param_list,
+ bool iser)
{
struct iscsi_param *param;
@@ -582,10 +583,15 @@ int iscsi_set_keys_irrelevant_for_discovery(
param-state = ~PSTATE_NEGOTIATE;
else if (!strcmp(param-name, RDMAEXTENSIONS))
param-state = ~PSTATE_NEGOTIATE;
- else if (!strcmp(param-name, INITIATORRECVDATASEGMENTLENGTH))
+ else if (!strcmp(param-name, INITIATORRECVDATASEGMENTLENGTH)) {
+ if (!iser)
+ continue;
param-state = ~PSTATE_NEGOTIATE;
- else if (!strcmp(param-name, TARGETRECVDATASEGMENTLENGTH))
+ } else if (!strcmp(param-name, TARGETRECVDATASEGMENTLENGTH)) {
+ if (!iser)
+ continue;
param-state = ~PSTATE_NEGOTIATE;
+ }
}
return 0;
diff --git a/drivers/target/iscsi/iscsi_target_parameters.h
b/drivers/target/iscsi/iscsi_target_para
index a47046a..76d2fdf 100644
--- a/drivers/target/iscsi/iscsi_target_parameters.h
+++ b/drivers/target/iscsi/iscsi_target_parameters.h
@@ -30,7 +30,7 @@ extern void iscsi_dump_sess_ops(struct iscsi_sess_ops *);
extern void iscsi_print_params(struct iscsi_param_list *);
extern int iscsi_create_default_params(struct iscsi_param_list **);
extern int iscsi_set_keys_to_negotiate(struct iscsi_param_list *, bool);
-extern int iscsi_set_keys_irrelevant_for_discovery(struct iscsi_param_list *);
+extern int iscsi_set_keys_irrelevant_for_discovery(struct iscsi_param_list *,
bool);
extern int iscsi_copy_param_list(struct
Re: do discovery through SW transports too
On 15/06/2013 11:29, Nicholas A. Bellinger wrote: Hi Or Mike, On Tue, 2013-06-11 at 08:04 +0300, Or Gerlitz wrote: On 06/06/2013 18:01, Mike Christie wrote: However, above I am not talking about that or doing discovery over a normal session in general. I was just trying to get clarification for what you wanted. I was not sure if there was some new iser spec stuff that I missed and you wanted to implement. Hi Mike, Its not new spec stuff, its just environments where plain TCP services might be out of hand for either of the sides and we don't want that to be an obstacle to deploy iser. No luck yet generating discovery ib_iser session traffic with mnc's initial patch + latest open-iscsi.git userspace. iscsiadm is currently defaulting to tcp for all sendtargets discovery: root@barret:# /sbin/iscsiadm -m discovery -t sendtargets --portal 10.100.0.1:3261 -I iser -d 4 iscsiadm: Max file limits 1024 1024 iscsiadm: Matched transport iser iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/handle' with attribute value '18446744072102379920' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/iser/caps' with attribute value '0x9' iscsiadm: Matched transport tcp iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'handle' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/handle' with attribute value '18446744072101696624' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'caps' iscsiadm: sysfs_attr_get_value: new uncached attribute '/sys/class/iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: add to cache '/sys/class/iscsi_transport/tcp/caps' iscsiadm: sysfs_attr_get_value: cache '/sys/class/iscsi_transport/tcp/caps' with attribute value '0x39' iscsiadm: Could not match iface[hw=,ip=,net_if=,iscsi_if=iser] to host. iscsiadm: starting sendtargets discovery, address 10.100.0.1:3261, iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/iser'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/iser/caps' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'handle' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/tcp/handle' iscsiadm: sysfs_attr_get_value: open '/class/iscsi_transport/tcp'/'caps' iscsiadm: sysfs_attr_get_value: found in cache '/class/iscsi_transport/tcp/caps' iscsiadm: iscsi_create_session discovery ep connect iser, caps: 0x0009 iscsiadm: connecting to 10.100.0.1:3261 iscsiadm: connected local port 60622 to 10.100.0.1:3261 iscsiadm: connected to discovery address 10.100.0.1 iscsiadm: login response status iscsiadm: discovery process to 10.100.0.1:3261 exiting iscsiadm: disconnecting conn 0x6d2f40, fd 3 192.168.1.69:3260,1 iqn.2003-01.org.linux-iscsi.tifa.x8664:sn.62d7638ae251 10.100.0.1:3260,1 iqn.foo 10.100.0.1:3261,1 iqn.foo 192.168.1.69:3260,1 iqn.foo 10.100.0.1:3260,1 iqn.foo 10.100.0.1:3261,1 iqn.foo 10.200.0.1:3260,1 iqn.foo2 10.200.0.1:3261,1 iqn.foo2 192.168.1.69:3260,1 iqn.foo2 10.200.0.1:3260,1 iqn.foo2 10.200.0.1:3261,1 iqn.foo2 Following the debug pointer, CAP_TEXT_NEGO appears to be missing from session-t-caps during iscsi_create_session() setup usage: iscsiadm: iscsi_create_session discovery ep connect iser, caps: 0x0009 Any reason why userspace would not matching iscsi_iser_transport-caps w/ CAP_TEXT_NEGO set (0x0089)..? Thanks! --nab Adding the list -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 11/06/2013 20:18, Mike Christie wrote: On 6/9/13 9:54 AM, Or Gerlitz wrote: usr]# make Hey, Did you figure this out? Support for that changed. You need to do make from the top level dir, because of dependencies on other stuff. Once that stuff is built you can do make from the usr dir to build just the usr stuff. OK, now iscsiadm and iscsid build fine! I am not using iscsistart but FWIW its build fails, cc -O2 -g -Wall -Wstrict-prototypes -I../include -I. -I../utils/open-isns -DLinux -DNETLINK_ISCSI=8 -D_GNU_SOURCE -static iscsi_util.o io.o auth.o iscsi_timer.o login.o log.o md5.o sha1.o iface.o idbm.o sysfs.o host.o session_info.o iscsi_sysfs.o iscsi_net_util.o iscsid_req.o transport.o iser.o cxgbi.o be2iscsi.o initiator_common.o iscsi_err.o flashnode.o uip_mgmt_ipc.o netlink.o ../utils/sysdeps/sysdeps.o initiator.o scsi.o actor.o event_poll.o mgmt_ipc.o kern_err_table.o ../utils/fwparam_ibft/fw_entry.o ../utils/fwparam_ibft/fwparam_ppc.o ../utils/fwparam_ibft/fwparam_sysfs.o ../utils/fwparam_ibft/prom_lex.o ../utils/fwparam_ibft/prom_parse.tab.o iscsistart.o statics.o -o iscsistart /usr/bin/ld: cannot find -lc -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Antw: Re: do discovery through SW transports too
Hi! I wonder: Isn't -static quite obsolete? If not, are you sure you installed the static C libraries? Regards, Ulrich Or Gerlitz ogerl...@mellanox.com schrieb am 12.06.2013 um 09:31 in Nachricht 51b823e6.5090...@mellanox.com: On 11/06/2013 20:18, Mike Christie wrote: On 6/9/13 9:54 AM, Or Gerlitz wrote: usr]# make Hey, Did you figure this out? Support for that changed. You need to do make from the top level dir, because of dependencies on other stuff. Once that stuff is built you can do make from the usr dir to build just the usr stuff. OK, now iscsiadm and iscsid build fine! I am not using iscsistart but FWIW its build fails, cc -O2 -g -Wall -Wstrict-prototypes -I../include -I. -I../utils/open-isns -DLinux -DNETLINK_ISCSI=8 -D_GNU_SOURCE -static iscsi_util.o io.o auth.o iscsi_timer.o login.o log.o md5.o sha1.o iface.o idbm.o sysfs.o host.o session_info.o iscsi_sysfs.o iscsi_net_util.o iscsid_req.o transport.o iser.o cxgbi.o be2iscsi.o initiator_common.o iscsi_err.o flashnode.o uip_mgmt_ipc.o netlink.o ../utils/sysdeps/sysdeps.o initiator.o scsi.o actor.o event_poll.o mgmt_ipc.o kern_err_table.o ../utils/fwparam_ibft/fw_entry.o ../utils/fwparam_ibft/fwparam_ppc.o ../utils/fwparam_ibft/fwparam_sysfs.o ../utils/fwparam_ibft/prom_lex.o ../utils/fwparam_ibft/prom_parse.tab.o iscsistart.o statics.o -o iscsistart /usr/bin/ld: cannot find -lc -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: Antw: Re: do discovery through SW transports too
On 12/06/2013 10:53, Ulrich Windl wrote: are you sure you installed the static C libraries? this might be the problem, I wasn't sure what rpm/s I need to install for that. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 6/9/13 9:54 AM, Or Gerlitz wrote: usr]# make Hey, Did you figure this out? Support for that changed. You need to do make from the top level dir, because of dependencies on other stuff. Once that stuff is built you can do make from the usr dir to build just the usr stuff. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/06/2013 18:01, Mike Christie wrote: However, above I am not talking about that or doing discovery over a normal session in general. I was just trying to get clarification for what you wanted. I was not sure if there was some new iser spec stuff that I missed and you wanted to implement. Hi Mike, Its not new spec stuff, its just environments where plain TCP services might be out of hand for either of the sides and we don't want that to be an obstacle to deploy iser. Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/06/2013 18:01, Mike Christie wrote: I think everything should be there. I thought I worked on iser when doing the work too. You need the attached kernel patch. In the other mails I think I said you need a change to the userspace iser code, but ignore that. In the common iscsi code I did a hack for all drivers. Apply the patch and then run iscsiadm -m discovery -t st -p ip -I iser -d 8 In the userspace discovery.c:iscsi_create_leading_conn() you will want to check that the code path with this debug log statement is being hit: log_debug(2, %s discovery ep connect\n, __FUNCTION__); and that it is the correct callout for your driver. Mike, I had some troubles building the user space part of the initiator from the latest git, FWYI - for utils/open-isns./configure --without-securityworks OK, but the default (which is --with-security) seems to be broken (see below). Also once, I passed that, I failed to get the usr directory utils to get built as of missing strlcpy/strlcat -- probably my glibc is too old and doesn't contain these folks... do you know of the latest initiator is buildable on RHEL 6.x nodes (e.g with glibc-2.12-1.47.el6.x86_64) ? auth.o: In function `acl_set_chap_alg_key': /open-iscsi/usr/auth.c:743: undefined reference to `strlcpy' /open-iscsi/usr/auth.c:746: undefined reference to `strlcat' another failure I see there is for the fw_xxx functions, any idea what should I build to make that work? discovery.o: In function `discovery_fw': /open-iscsi/usr/discovery.c:391: undefined reference to `fw_get_targets' /open-iscsi/usr/discovery.c:419: undefined reference to `fw_free_targets' Or. checking openssl/crypto.h usability... yes checking openssl/crypto.h presence... yes checking for openssl/crypto.h... yes checking for EVP_PKEY_new in -lcrypto... yes checking slp.h usability... no checking slp.h presence... no checking for slp.h... no checking for SLPOpen in -lslp... no configure: creating ./config.status config.status: creating Makefile config.status: WARNING: 'Makefile.in' seems to ignore the --datarootdir setting config.status: creating config.h config.status: config.h is unchanged [root@vsa3 open-isns]# make gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o server.o server.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o client.o client.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o objects.o objects.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o callback.o callback.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o timer.o timer.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o vendor.o vendor.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o db.o db.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o db-file.o db-file.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o db-policy.o db-policy.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o relation.o relation.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o scope.o scope.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o message.o message.c gcc -Wall -fno-strict-aliasing -g -O2 -I. -D_GNU_SOURCE -c -o security.o security.c security.c: In function ?isns_create_principal?: security.c:40: warning: dereferencing ?void *? pointer security.c:40: error: request for member ?type? in something not a structure or union security.c:41: error: ?EVP_PKEY_DSA? undeclared (first use in this function) security.c:41: error: (Each undeclared identifier is reported only once security.c:41: error: for each function it appears in.) security.c:42: error: ?EVP_PKEY_RSA? undeclared (first use in this function) security.c:47: warning: implicit declaration of function ?EVP_PKEY_bits? security.c: In function ?isns_principal_set_key?: security.c:61: warning: implicit declaration of function ?EVP_PKEY_free? make: *** [security.o] Error 1 usr]# make cc -O2 -g -Wall -Wstrict-prototypes -I../include -I. -I../utils/open-isns -DLinux -DNETLINK_ISCSI=8 -D_GNU_SOURCE iscsi_util.o io.o auth.o iscsi_timer.o login.o log.o md5.o sha1.o iface.o idbm.o sysfs.o host.o session_info.o iscsi_sysfs.o iscsi_net_util.o iscsid_req.o transport.o iser.o cxgbi.o be2iscsi.o initiator_common.o iscsi_err.o flashnode.o uip_mgmt_ipc.o netlink.o initiator.o scsi.o actor.o event_poll.o mgmt_ipc.o kern_err_table.o strings.o discovery.o iscsid.o session_mgmt.o discoveryd.o -o iscsid -L../utils/open-isns -lisns io.o: In function `bind_conn_to_iface': /mnt/sdb1/ogerlitz/open-iscsi/usr/io.c:237: undefined reference to `strlcpy' auth.o: In function `acl_set_key_value': /mnt/sdb1/ogerlitz/open-iscsi/usr/auth.c:547: undefined reference to `strlcpy' auth.o: In function `acl_set_user_name': /mnt/sdb1/ogerlitz/open-iscsi/usr/auth.c:1890: undefined reference to `strlcpy' auth.o: In function `acl_send_key_val': /mnt/sdb1/ogerlitz/open-iscsi/usr/auth.c:1520:
Re: do discovery through SW transports too
On Thu, Jun 6, 2013 at 2:16 AM, Mike Christie micha...@cs.wisc.edu wrote: On 06/05/2013 03:36 PM, Or Gerlitz wrote: Mike Christie micha...@cs.wisc.edu wrote: Mike, Thanks for clarifying things out here, calls the transport template ep_connect/poll/disconnect and send_pdu calls likes is done for normal session login. That code then will do discovery through that connection. To use this the driver only needs to set CAP_TEXT_NEGO sounds very promising, but for and then also have code to support discovery commands. I need some clarifications, if user space does send pdu and we have mechanisms in place to delivered a pdu received by the kernel to user space. Then from the iser kernel transport view point running a discovery session is an iscsi/iser session without SCSI commands just login, some text PDUs and logout. To be precise 1. we don't want to offload the sendtargets based discovery but rather to conduct it over an iser IB/RoCE connection and not TCP connection I am not sure what you are talking about for offload. Are you calling what the iser kernel module does offload? If so why is that? Just wondering, because everyone has different definitions. Is it because of zero copy? I was thinking it just creates some rdma connection in the ep_connect callbacks then we throw pdus over it to create the session, and I was not considering this offload. You do not want that right? Why is that? Yes, create IB/RoCE connection and throwing the sendtargets PDUs over it If you do not want to do that then do you want to do this in the kernel or userspace? We can do in either of them, but I would love to just re-use the kernel code. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On Thu, Jun 6, 2013 at 2:22 AM, Mike Christie micha...@cs.wisc.edu wrote: On 06/05/2013 03:36 PM, Or Gerlitz wrote: 1. we don't want to offload the sendtargets based discovery but rather to conduct it over an iser IB/RoCE connection and not TCP connection One other question/clarification. You do discovery over a iscsi session. It is a discovery session instead of a normal session. So we cannot just create a connection then send pdus over it. We have to go through the entire process of creating a session. When we login we negotiate that it is a discovery session. I think it would be best to aim for iscsi discovery session which is conducted over IB/RoCE connection opened by the iser transport. E.g allow for both discovery/normal sessions to be carried out over the iser transport. Do we have most/everything in place to makde that happen? if not, what is missing, need some coach here. For your #1, it was not clear if you are also planning on doing normal old iscsi discovery but over a ib/rcoe connection or if you are doing something really new like discovery over only a ib/rcoe connection (no iscsi session in this case). From the target view point, I am not sure what does it mean do discovery over normal session, is that possible spec wise? Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/06/2013 01:28 AM, Or Gerlitz wrote:
On Thu, Jun 6, 2013 at 2:22 AM, Mike Christie micha...@cs.wisc.edu wrote:
On 06/05/2013 03:36 PM, Or Gerlitz wrote:
1. we don't want to offload the sendtargets based discovery but rather
to conduct it over an iser IB/RoCE connection and not TCP connection
One other question/clarification. You do discovery over a iscsi session.
It is a discovery session instead of a normal session. So we cannot just
create a connection then send pdus over it. We have to go through the
entire process of creating a session. When we login we negotiate that it
is a discovery session.
I think it would be best to aim for iscsi discovery session which is
conducted over IB/RoCE connection opened by the iser transport. E.g
allow for both discovery/normal sessions to be carried out over the
iser transport. Do we have most/everything in place to makde that
happen? if not, what is missing, need some coach here.
I think everything should be there. I thought I worked on iser when
doing the work too. You need the attached kernel patch.
In the other mails I think I said you need a change to the userspace
iser code, but ignore that. In the common iscsi code I did a hack for
all drivers.
Apply the patch and then run
iscsiadm -m discovery -t st -p ip -I iser -d 8
In the userspace discovery.c:iscsi_create_leading_conn() you will want
to check that the code path with this debug log statement is being hit:
log_debug(2, %s discovery ep connect\n, __FUNCTION__);
and that it is the correct callout for your driver.
For your #1, it was not clear if you are also planning on doing normal
old iscsi discovery but over a ib/rcoe connection or if you are doing
something really new like discovery over only a ib/rcoe connection (no
iscsi session in this case).
From the target view point, I am not sure what does it mean do
discovery over normal session, is that possible spec wise?
Sort of. You can send sendtargets in a normal session. See appendix D of
the RFC. If it is sent during a normal session I think you only get path
info for the target the session is connected to. You do not get all
targets like if you send it during a discovery session. I think some
initiator does it or was trying to do it when getting some sense code
indicating something changed (I cannot remember it off the top of my
head). So it is kinda neat that way if you are dynamically
adding/deleting paths and want the init to handle that automatically.
However, above I am not talking about that or doing discovery over a
normal session in general. I was just trying to get clarification for
what you wanted. I was not sure if there was some new iser spec stuff
that I missed and you wanted to implement.
--
You received this message because you are subscribed to the Google Groups
open-iscsi group.
To unsubscribe from this group and stop receiving emails from it, send an email
to open-iscsi+unsubscr...@googlegroups.com.
To post to this group, send email to open-iscsi@googlegroups.com.
Visit this group at http://groups.google.com/group/open-iscsi?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
diff --git a/drivers/infiniband/ulp/iser/iscsi_iser.c
b/drivers/infiniband/ulp/iser/iscsi_iser.c
index f19b099..42c8ff4 100644
--- a/drivers/infiniband/ulp/iser/iscsi_iser.c
+++ b/drivers/infiniband/ulp/iser/iscsi_iser.c
@@ -700,7 +700,8 @@ static struct scsi_host_template iscsi_iser_sht = {
static struct iscsi_transport iscsi_iser_transport = {
.owner = THIS_MODULE,
.name = iser,
- .caps = CAP_RECOVERY_L0 | CAP_MULTI_R2T,
+ .caps = CAP_RECOVERY_L0 | CAP_MULTI_R2T |
+ CAP_TEXT_NEGO,
/* session management */
.create_session = iscsi_iser_session_create,
.destroy_session= iscsi_iser_session_destroy,
Re: do discovery through SW transports too
On Thu, Jun 6, 2013 at 6:01 PM, Mike Christie micha...@cs.wisc.edu wrote: On 06/06/2013 01:28 AM, Or Gerlitz wrote: I think everything should be there. I thought I worked on iser when doing the work too. You need the attached kernel patch. In the other mails I think I said you need a change to the userspace iser code, but ignore that. In the common iscsi code I did a hack for all drivers. Apply the patch and then run iscsiadm -m discovery -t st -p ip -I iser -d 8 cool^2, will try that out Sunday, just to dounle check, this would use discovery session over the iser transport, not normal sesssion, correct? In the userspace discovery.c:iscsi_create_leading_conn() you will want to check that the code path with this debug log statement is being hit: log_debug(2, %s discovery ep connect\n, __FUNCTION__); and that it is the correct callout for your driver. For your #1, it was not clear if you are also planning on doing normal old iscsi discovery but over a ib/rcoe connection or if you are doing something really new like discovery over only a ib/rcoe connection (no iscsi session in this case). From the target view point, I am not sure what does it mean do discovery over normal session, is that possible spec wise? Sort of. You can send sendtargets in a normal session. See appendix D of the RFC. If it is sent during a normal session I think you only get path info for the target the session is connected to. You do not get all targets like if you send it during a discovery session. I think some indeed, we want discovery session over iser not sending sendtargets pdu in normal session. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/05/2013 11:35 AM, Or Gerlitz wrote: Hi Mike, There are iser environments where we might need to do iscsi discovery over rdma connection, that is establish iser connection and then carry the discovery session over it. I was a bit away for the developments in the initiator over the last months so would need some crash help here. Looking on the user space code, I see that if a transport advertizes SENDTARGETS_OFFLOAD capability there's possibility for them to provide -sendtargets entry through which discovery can be done. I also see in usr/netlink.c framework to go into the kernel for sendtargets. I understand this was build for HW iscsi transports where the card offloads things like discovery, but I hope we can reuse this code if the user wants to (say) conduct discovery through any transport (e.g iser) and not through tcp from user space, any heads up will be cool. I made bnx2i/cxgb*i/be2iscsi/qlogic-session-node support support this. I think iser needs some tiny changes. I thought I did iser but do not see the patches on the list. If you have to go to the kernel and you need to do some ep_connect type of calls like with normal sessions then most of that work should be done for you. Right now the userspace discovery.c code calls the transport template ep_connect/poll/disconnect and send_pdu calls call likes is done for normal session login. That code then will do discovery through that connection. To use this the driver only needs to set CAP_TEXT_NEGO and then also have code to support discovery commands. Since you use libiscsi then the higher level iscsi discovery stuff should be done there already. I cannot remember if iser needed some changes to support this. I think the iser driver needed to set CAP_TEXT_NEGO. Also then in the userspace iser.c file you need something to make sure the max_xmit_dlength/max_recv_dlength is 8k because the kernel buffer used for login is limited to that size. Do you actually want to be able to try either rdma or tcp? If so then I think you will need more code. I do not have support for some sort of discovery failover. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
Mike Christie micha...@cs.wisc.edu wrote: Mike, Thanks for clarifying things out here, calls the transport template ep_connect/poll/disconnect and send_pdu calls likes is done for normal session login. That code then will do discovery through that connection. To use this the driver only needs to set CAP_TEXT_NEGO sounds very promising, but for and then also have code to support discovery commands. I need some clarifications, if user space does send pdu and we have mechanisms in place to delivered a pdu received by the kernel to user space. Then from the iser kernel transport view point running a discovery session is an iscsi/iser session without SCSI commands just login, some text PDUs and logout. To be precise 1. we don't want to offload the sendtargets based discovery but rather to conduct it over an iser IB/RoCE connection and not TCP connection 2. we need a way for the user to config if sendtargets will be over tcp connection or over iser connection. I don't think we need fallback. How would you recommend (e.g patches/changes/field tries... we need to do) to get this progressing? Or. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/05/2013 03:36 PM, Or Gerlitz wrote: Mike Christie micha...@cs.wisc.edu wrote: Mike, Thanks for clarifying things out here, calls the transport template ep_connect/poll/disconnect and send_pdu calls likes is done for normal session login. That code then will do discovery through that connection. To use this the driver only needs to set CAP_TEXT_NEGO sounds very promising, but for and then also have code to support discovery commands. I need some clarifications, if user space does send pdu and we have mechanisms in place to delivered a pdu received by the kernel to user space. Then from the iser kernel transport view point running a discovery session is an iscsi/iser session without SCSI commands just login, some text PDUs and logout. To be precise 1. we don't want to offload the sendtargets based discovery but rather to conduct it over an iser IB/RoCE connection and not TCP connection I am not sure what you are talking about for offload. Are you calling what the iser kernel module does offload? If so why is that? Just wondering, because everyone has different definitions. Is it because of zero copy? I was thinking it just creates some rdma connection in the ep_connect callbacks then we throw pdus over it to create the session, and I was not considering this offload. You do not want that right? Why is that? If you do not want to do that then do you want to do this in the kernel or userspace? If kernel then tell me how it is different from above? If userspace then you would want to modify the transport.c code with some new callbacks for discovery. In your callbacks you would want to do your userspace connection stuff. Then in discovery.c you would want to modify it to call these new discovery ep callbacks. So you might add: disc_ep_connect disc_ep_poll disc_ep_disconnect For the other drivers those would point to what they point to for their existing ep functions. For iser they would point to some new functions. Then in discovery.c you would just have that code call to the new disc_ep_* functions. 2. we need a way for the user to config if sendtargets will be over tcp connection or over iser connection. I don't think we need fallback. Yeah. Wrote that too quick. I was thinking you want something similar in that you want some way to tell users to use different code paths for discovery. -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: do discovery through SW transports too
On 06/05/2013 03:36 PM, Or Gerlitz wrote: I need some clarifications, if user space does send pdu and we have mechanisms in place to delivered a pdu received by the kernel to user space. Then from the iser kernel transport view point running a discovery session is an iscsi/iser session without SCSI commands just login, some text PDUs and logout. To be precise 1. we don't want to offload the sendtargets based discovery but rather to conduct it over an iser IB/RoCE connection and not TCP connection One other question/clarification. You do discovery over a iscsi session. It is a discovery session instead of a normal session. So we cannot just create a connection then send pdus over it. We have to go through the entire process of creating a session. When we login we negotiate that it is a discovery session. For your #1, it was not clear if you are also planning on doing normal old iscsi discovery but over a ib/rcoe connection or if you are doing something really new like discovery over only a ib/rcoe connection (no iscsi session in this case). -- You received this message because you are subscribed to the Google Groups open-iscsi group. To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscr...@googlegroups.com. To post to this group, send email to open-iscsi@googlegroups.com. Visit this group at http://groups.google.com/group/open-iscsi?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
