RE: [Scst-devel] scst qla2xxx.ko VS kernel qla2xxx.ko
Greg . Thank you very much for the detailed mail . I will mail you and the list for any further quires to the list . Thanks. /Suresh -Original Message- From: Dr. Greg Wettstein [mailto:g...@wind.enjellic.com] Sent: Saturday, March 21, 2015 1:36 AM To: Suresh Babu Kandukuru; Bart Van Assche; scst-de...@lists.sourceforge.net Cc: linux-scsi@vger.kernel.org Subject: Re: [Scst-devel] scst qla2xxx.ko VS kernel qla2xxx.ko On Mar 13, 4:18am, Suresh Babu Kandukuru wrote: } Subject: Re: [Scst-devel] scst qla2xxx.ko VS kernel qla2xxx.ko Hi, I hope the week is closing out well for everyone. Thanks Bart . We have tested with 2.2.0 and 3.0.1 (scst-svn-6135-branches-3.0.x ) ., loaded all the scst ko files along with qla2xxx_scst.ko qla2x00t.ko . Still this did not any FC ports in /sys/class/fc_hosts/.. . The QLOGIC HBA we are using is QLogic QLE8362 - 7101674, Sun Storage 16Gb FC PCIe Universal HBA, QLogic. . Does the scst qla2xxx_scst.ko driver supports this ?. what we are missing here ?. Any pointers are helpful . Thanks We have had the opportunity to architect, build, maintain and debug complex converged data and storage networks. The switches on these networks are a million dollars apiece and there are multiple switches in multiple data-centers. The storage, running at about one-half petabyte levels, is all SCST based and we use Qlogic HBA's and CNA's on both the initiators and targets. Given the somewhat colorful history with respect to Qlogic target driver support we had to roll up our sleeves and write an interface driver which allows SCST to run on top of the Qlogic target driver in stock Linux kernels. I just put an integrated patchset which includes SCST 3.0.0, our target interface driver and a few patches to the in-kernel target driver we carry to make all of this work correctly and address various regressions in the stock target driver. You can grab the patchset from the following URL: ftp://ftp.enjellic.com/pub/scst/sqa-1.2_scst-3.0.0_3.10.67.patch.gz All of the storage targets are direct FCOE attachments using Qlogic 8362 CNA's. The above patchset is what we are currently using in production and it has been validated over hundreds and hundreds of terrabytes of I/O, which includes exposure to some rather rigorous fabric and TMF event processing situations. We have also tracked down some rather elusive switch based FCOE transport issues with this codebase. Obviously your mileage may vary. If you drop the above patch onto a 3.10 kernel source tree and build a kernel with the appropriate configuration options it should work with whatever cards are supported by the stock Linux kernel. One of the patches which is included sets the card to QLA2XXX_INI_MODE_EXCLUSIVE which causes the card to come up 'target capable' by default which mirrors the behavior of the SCST supplied Qlogic target driver. In your note you indicate your card is a 'QLogic QLE8362 - 7101674, Sun Storage 16Gb FC PCIe Universal HBA' which I assume is probably an OEM rebranded card. The 8362 is actually a Converged Network Adapter (CNA) rather then an HBA. The 2672 is the 'fibre-channel' HBA version of the card. The stock Qlogic cards are dual-personality cards which allows their operational mode, ie 16 GBPS FC or 10 GBPS ethernet, to be selected through the BIOS setup utilities. Given your e-mail address, I'm sure you've jumped through all these hoops but one obviously wants to have the card setup with the right personality and correct optics for your physical transport layer. You also probably want to make sure you have both your HBA/CNA and motherboard up to the most recent versions of their respective firmware. There are issues particularly with Intel S2600 motherboards and these cards which requires updated firmware on both sides. If the card is detected and found to be target capable by the stock Linux kernel Qlogic driver you will find the available target PWWN's in the following directory: /sys/kernel/scst_tgt/targets/sqatgt If there are none present there is the possibility that the card has been modified in firmware to prohibit target mode or has a modified PCI id which is causing problems for the driver probe logic. If that is the case the card isn't going to be a candidate for use in target mode. We tend to recommend that people buy straight Qlogic branded cards to avoid these sorts of issues. Just as an FYI/CYA there are session management issues/bugs in the Qlogic target driver, regardless of breed; SCST, Qlogic SCST or kernel. These can show up during asynchronous fabric processing events, particularly if you are aggressive in placing the card and fabric under duress. We are in the process of upgrading our patchset to 3.0.1 which will include a number of additional fixes to the in-kernel Qlogic target driver. I will shoot a note off to the list when it is available if you or others find this stuff useful. I hope the above is
[PATCH 0/3] iscsi/iser-target: Convert to kthread.h + fix logout failure
From: Nicholas Bellinger n...@linux-iscsi.org Hello Sagi Slava, Following the earlier pre-LSF discussion, here is a patch series to convert iscsi/iser target to using modern kthread.h primitives when starting / stopping RX + TX kernel threads during iscsi_conn creation / shutdown. So far this code has been tested with traditional iscsi-target login / logout code-paths, and needs to be verified via iser-target login / logout code-paths as well. Also, patch #3 is a seperate iser-target bugfix to handle a failed outgoing ISCSI_OP_LOGOUT_RSP in isert_cq_comp_err() completion interrupt path code. This one may or may not have been triggering earlier, but addresses a possible iser-target specific logout bug. Please review + test for v4.1-rc1 code. Thank you, --nab Nicholas Bellinger (3): iscsi-target: Convert iscsi_thread_set usage to kthread.h iscsi-target: Drop legacy iscsi_target_tq.c logic iser-target: Handle special case for logout during connection failure drivers/infiniband/ulp/isert/ib_isert.c | 16 +- drivers/target/iscsi/Makefile | 1 - drivers/target/iscsi/iscsi_target.c | 109 +++ drivers/target/iscsi/iscsi_target_erl0.c | 14 +- drivers/target/iscsi/iscsi_target_login.c | 60 +++- drivers/target/iscsi/iscsi_target_tq.c| 495 -- drivers/target/iscsi/iscsi_target_tq.h| 84 - drivers/target/iscsi/iscsi_target_util.c | 1 - include/target/iscsi/iscsi_target_core.h | 13 +- 9 files changed, 131 insertions(+), 662 deletions(-) delete mode 100644 drivers/target/iscsi/iscsi_target_tq.c delete mode 100644 drivers/target/iscsi/iscsi_target_tq.h -- 1.9.1 -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3] iscsi-target: Drop legacy iscsi_target_tq.c logic
From: Nicholas Bellinger n...@linux-iscsi.org
Now that iscsi_conn allocates new [rx,tx] threads using kthread.h
primitives on the fly, and kthread_stop() is called directly during
connection shutdown, it's time to go ahead and drop iscsi_target_tq.c
legacy code.
The use of multiple struct completion in iscsi_activate_thread_set()
has been proven to cause issues during repeated iser login/logout.
Cc: Sagi Grimberg sa...@mellanox.com
Cc: Slava Shwartsman valyush...@gmail.com
Signed-off-by: Nicholas Bellinger n...@linux-iscsi.org
---
drivers/target/iscsi/Makefile | 1 -
drivers/target/iscsi/iscsi_target.c | 3 -
drivers/target/iscsi/iscsi_target_erl0.c | 1 -
drivers/target/iscsi/iscsi_target_login.c | 1 -
drivers/target/iscsi/iscsi_target_tq.c| 495 --
drivers/target/iscsi/iscsi_target_tq.h| 84 -
drivers/target/iscsi/iscsi_target_util.c | 1 -
include/target/iscsi/iscsi_target_core.h | 6 +-
8 files changed, 2 insertions(+), 590 deletions(-)
delete mode 100644 drivers/target/iscsi/iscsi_target_tq.c
delete mode 100644 drivers/target/iscsi/iscsi_target_tq.h
diff --git a/drivers/target/iscsi/Makefile b/drivers/target/iscsi/Makefile
index 13a9240..0f43be9 100644
--- a/drivers/target/iscsi/Makefile
+++ b/drivers/target/iscsi/Makefile
@@ -1,6 +1,5 @@
iscsi_target_mod-y += iscsi_target_parameters.o \
iscsi_target_seq_pdu_list.o \
- iscsi_target_tq.o \
iscsi_target_auth.o \
iscsi_target_datain_values.o \
iscsi_target_device.o \
diff --git a/drivers/target/iscsi/iscsi_target.c
b/drivers/target/iscsi/iscsi_target.c
index cb97b59..ca731b5 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -33,7 +33,6 @@
#include target/iscsi/iscsi_target_core.h
#include iscsi_target_parameters.h
#include iscsi_target_seq_pdu_list.h
-#include iscsi_target_tq.h
#include iscsi_target_configfs.h
#include iscsi_target_datain_values.h
#include iscsi_target_erl0.h
@@ -4369,8 +4368,6 @@ int iscsit_close_connection(
iscsit_put_transport(conn-conn_transport);
- conn-thread_set = NULL;
-
pr_debug(Moving to TARG_CONN_STATE_FREE.\n);
conn-conn_state = TARG_CONN_STATE_FREE;
kfree(conn);
diff --git a/drivers/target/iscsi/iscsi_target_erl0.c
b/drivers/target/iscsi/iscsi_target_erl0.c
index e008ed2..959a14c 100644
--- a/drivers/target/iscsi/iscsi_target_erl0.c
+++ b/drivers/target/iscsi/iscsi_target_erl0.c
@@ -23,7 +23,6 @@
#include target/iscsi/iscsi_target_core.h
#include iscsi_target_seq_pdu_list.h
-#include iscsi_target_tq.h
#include iscsi_target_erl0.h
#include iscsi_target_erl1.h
#include iscsi_target_erl2.h
diff --git a/drivers/target/iscsi/iscsi_target_login.c
b/drivers/target/iscsi/iscsi_target_login.c
index 345f073..af20ddf 100644
--- a/drivers/target/iscsi/iscsi_target_login.c
+++ b/drivers/target/iscsi/iscsi_target_login.c
@@ -26,7 +26,6 @@
#include target/iscsi/iscsi_target_core.h
#include target/iscsi/iscsi_target_stat.h
-#include iscsi_target_tq.h
#include iscsi_target_device.h
#include iscsi_target_nego.h
#include iscsi_target_erl0.h
diff --git a/drivers/target/iscsi/iscsi_target_tq.c
b/drivers/target/iscsi/iscsi_target_tq.c
deleted file mode 100644
index 26aa509..000
--- a/drivers/target/iscsi/iscsi_target_tq.c
+++ /dev/null
@@ -1,495 +0,0 @@
-/***
- * This file contains the iSCSI Login Thread and Thread Queue functions.
- *
- * (c) Copyright 2007-2013 Datera, Inc.
- *
- * Author: Nicholas A. Bellinger n...@linux-iscsi.org
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
-
**/
-
-#include linux/kthread.h
-#include linux/list.h
-#include linux/bitmap.h
-
-#include target/iscsi/iscsi_target_core.h
-#include iscsi_target_tq.h
-#include iscsi_target.h
-
-static LIST_HEAD(inactive_ts_list);
-static DEFINE_SPINLOCK(inactive_ts_lock);
-static DEFINE_SPINLOCK(ts_bitmap_lock);
-
-static void iscsi_add_ts_to_inactive_list(struct iscsi_thread_set *ts)
-{
- if (!list_empty(ts-ts_list)) {
- WARN_ON(1);
- return;
- }
- spin_lock(inactive_ts_lock);
- list_add_tail(ts-ts_list, inactive_ts_list);
- iscsit_global-inactive_ts++;
-
[PATCH 1/3] iscsi-target: Convert iscsi_thread_set usage to kthread.h
From: Nicholas Bellinger n...@linux-iscsi.org
This patch converts iscsi-target code to use modern kthread.h API
callers for creating RX/TX threads for each new iscsi_conn descriptor,
and releasing associated RX/TX threads during connection shutdown.
This is done using iscsit_start_kthreads() - kthread_run() to start
new kthreads from within iscsi_post_login_handler(), and invoking
kthread_stop() from existing iscsit_close_connection() code.
Also, convert iscsit_logout_post_handler_closesession() code to use
cmpxchg when determing when iscsit_cause_connection_reinstatement()
needs to sleep waiting for completion.
Reported-by: Sagi Grimberg sa...@mellanox.com
Cc: Sagi Grimberg sa...@mellanox.com
Cc: Slava Shwartsman valyush...@gmail.com
Signed-off-by: Nicholas Bellinger n...@linux-iscsi.org
---
drivers/target/iscsi/iscsi_target.c | 106 +-
drivers/target/iscsi/iscsi_target_erl0.c | 13 ++--
drivers/target/iscsi/iscsi_target_login.c | 59 +++--
include/target/iscsi/iscsi_target_core.h | 7 ++
4 files changed, 116 insertions(+), 69 deletions(-)
diff --git a/drivers/target/iscsi/iscsi_target.c
b/drivers/target/iscsi/iscsi_target.c
index 2accb6e..cb97b59 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -537,7 +537,7 @@ static struct iscsit_transport iscsi_target_transport = {
static int __init iscsi_target_init_module(void)
{
- int ret = 0;
+ int ret = 0, size;
pr_debug(iSCSI-Target ISCSIT_VERSION\n);
@@ -546,6 +546,7 @@ static int __init iscsi_target_init_module(void)
pr_err(Unable to allocate memory for iscsit_global\n);
return -1;
}
+ spin_lock_init(iscsit_global-ts_bitmap_lock);
mutex_init(auth_id_lock);
spin_lock_init(sess_idr_lock);
idr_init(tiqn_idr);
@@ -555,15 +556,11 @@ static int __init iscsi_target_init_module(void)
if (ret 0)
goto out;
- ret = iscsi_thread_set_init();
- if (ret 0)
+ size = BITS_TO_LONGS(ISCSIT_BITMAP_BITS) * sizeof(long);
+ iscsit_global-ts_bitmap = vzalloc(size);
+ if (!iscsit_global-ts_bitmap) {
+ pr_err(Unable to allocate iscsit_global-ts_bitmap\n);
goto configfs_out;
-
- if (iscsi_allocate_thread_sets(TARGET_THREAD_SET_COUNT) !=
- TARGET_THREAD_SET_COUNT) {
- pr_err(iscsi_allocate_thread_sets() returned
-unexpected value!\n);
- goto ts_out1;
}
lio_qr_cache = kmem_cache_create(lio_qr_cache,
@@ -572,7 +569,7 @@ static int __init iscsi_target_init_module(void)
if (!lio_qr_cache) {
pr_err(nable to kmem_cache_create() for
lio_qr_cache\n);
- goto ts_out2;
+ goto bitmap_out;
}
lio_dr_cache = kmem_cache_create(lio_dr_cache,
@@ -617,10 +614,8 @@ dr_out:
kmem_cache_destroy(lio_dr_cache);
qr_out:
kmem_cache_destroy(lio_qr_cache);
-ts_out2:
- iscsi_deallocate_thread_sets();
-ts_out1:
- iscsi_thread_set_free();
+bitmap_out:
+ vfree(iscsit_global-ts_bitmap);
configfs_out:
iscsi_target_deregister_configfs();
out:
@@ -630,8 +625,6 @@ out:
static void __exit iscsi_target_cleanup_module(void)
{
- iscsi_deallocate_thread_sets();
- iscsi_thread_set_free();
iscsit_release_discovery_tpg();
iscsit_unregister_transport(iscsi_target_transport);
kmem_cache_destroy(lio_qr_cache);
@@ -641,6 +634,7 @@ static void __exit iscsi_target_cleanup_module(void)
iscsi_target_deregister_configfs();
+ vfree(iscsit_global-ts_bitmap);
kfree(iscsit_global);
}
@@ -3710,17 +3704,16 @@ static int iscsit_send_reject(
void iscsit_thread_get_cpumask(struct iscsi_conn *conn)
{
- struct iscsi_thread_set *ts = conn-thread_set;
int ord, cpu;
/*
-* thread_id is assigned from iscsit_global-ts_bitmap from
-* within iscsi_thread_set.c:iscsi_allocate_thread_sets()
+* bitmap_id is assigned from iscsit_global-ts_bitmap from
+* within iscsit_start_kthreads()
*
-* Here we use thread_id to determine which CPU that this
-* iSCSI connection's iscsi_thread_set will be scheduled to
+* Here we use bitmap_id to determine which CPU that this
+* iSCSI connection's RX/TX threads will be scheduled to
* execute upon.
*/
- ord = ts-thread_id % cpumask_weight(cpu_online_mask);
+ ord = conn-bitmap_id % cpumask_weight(cpu_online_mask);
for_each_online_cpu(cpu) {
if (ord-- == 0) {
cpumask_set_cpu(cpu, conn-conn_cpumask);
@@ -3909,7 +3902,7 @@ check_rsp_state:
switch (state) {
case ISTATE_SEND_LOGOUTRSP:
if (!iscsit_logout_post_handler(cmd, conn))
-
[PATCH 3/3] iser-target: Handle special case for logout during connection failure
From: Nicholas Bellinger n...@linux-iscsi.org
This patch adds special case handling during ISCSI_OP_LOGIN_RSP ib_wr
failure, where isert_cq_comp_err() is responsible for calling the
remaining isert_response_completion() - isert_do_control_comp() -
iscsit_logout_post_handler() to drop the last iscsi_conn reference.
It fixes a bug where iscsit_logout_post_handler() would not be called
if the outgoing ISCSI_OP_LOGIN_RSP failed during iscsi_conn ib_wr
descriptor cleanup.
Reported-by: Sagi Grimberg sa...@mellanox.com
Reported-by: Slava Shwartsman valyush...@gmail.com
Signed-off-by: Nicholas Bellinger n...@linux-iscsi.org
---
drivers/infiniband/ulp/isert/ib_isert.c | 16 +---
1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/ulp/isert/ib_isert.c
b/drivers/infiniband/ulp/isert/ib_isert.c
index 075b19c..ff0d436 100644
--- a/drivers/infiniband/ulp/isert/ib_isert.c
+++ b/drivers/infiniband/ulp/isert/ib_isert.c
@@ -1996,10 +1996,20 @@ isert_cq_comp_err(struct isert_conn *isert_conn, struct
ib_wc *wc)
desc = (struct iser_tx_desc *)(uintptr_t)wc-wr_id;
isert_cmd = desc-isert_cmd;
- if (!isert_cmd)
+ if (!isert_cmd) {
isert_unmap_tx_desc(desc, ib_dev);
- else
- isert_completion_put(desc, isert_cmd, ib_dev, true);
+ } else {
+ struct isert_device *device = isert_conn-conn_device;
+ struct iscsi_conn *conn = isert_conn-conn;
+ struct iscsi_cmd *cmd = isert_cmd-iscsi_cmd;
+
+ if (cmd-i_state == ISTATE_SEND_LOGOUTRSP
+ conn-conn_state == TARG_CONN_STATE_IN_LOGOUT)
+ isert_response_completion(desc, isert_cmd,
isert_conn,
+ device-ib_device);
+ else
+ isert_completion_put(desc, isert_cmd, ib_dev,
true);
+ }
} else {
isert_conn-post_recv_buf_count--;
if (!isert_conn-post_recv_buf_count)
--
1.9.1
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 0/3] scsi: storvsc: Increase the tablesize based on host's capabilities
-Original Message- From: KY Srinivasan Sent: Monday, March 16, 2015 4:45 PM To: 'K. Y. Srinivasan'; gre...@linuxfoundation.org; linux- ker...@vger.kernel.org; de...@linuxdriverproject.org; oher...@suse.com; jbottom...@parallels.com; h...@infradead.org; linux-scsi@vger.kernel.org; a...@canonical.com; vkuzn...@redhat.com Subject: RE: [PATCH 0/3] scsi: storvsc: Increase the tablesize based on host's capabilities -Original Message- From: K. Y. Srinivasan [mailto:k...@microsoft.com] Sent: Monday, March 9, 2015 8:42 PM To: gre...@linuxfoundation.org; linux-ker...@vger.kernel.org; de...@linuxdriverproject.org; oher...@suse.com; jbottom...@parallels.com; h...@infradead.org; linux-scsi@vger.kernel.org; a...@canonical.com; vkuzn...@redhat.com Cc: KY Srinivasan Subject: [PATCH 0/3] scsi: storvsc: Increase the tablesize based on host's capabilities Presently, storvsc limits the I/O size arbitrarily. Make this configurable based on what the host advertises. K. Y. Srinivasan (3): scsi: storvsc: Retrieve information about the capability of the target scsi: storvsc: Set the tablesize based on the information given by the host scsi: storvsc: Enable clustering drivers/scsi/storvsc_drv.c | 78 +++- 1 files changed, 62 insertions(+), 16 deletions(-) Christoph, Let me know if I should resend these patches. Regards, Christoph, Please drop this set. I will resend the patches with a couple of additional patches that need to go in before I can enable clustering. Regards, K. Y -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
