Re: [PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-08-03 Thread Sreekanth Reddy 
On Thu, Aug 3, 2017 at 11:57 AM, Hannes Reinecke  wrote:
> On 07/14/2017 03:22 PM, Suganath Prabu S wrote:
>> 1) Added support for probing pcie device and adding NVMe drives to
>> SML and driver's internal list pcie_device_list.
>>
>> 2) Added support for determing NVMe as boot device.
>>
>> 3) Added nvme device support for call back functions scan_finished
>> target_alloc,slave_alloc,target destroy and slave destroy.
>>
>>  a) During scan, pcie devices are probed and added to SML to drivers
>> internal list.
>>
>>  b) target_alloc & slave alloc API's allocates resources for
>> (MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
>> information like handle, target_id etc.
>>
>>  c) slave_destroy & target_destroy are called when driver unregisters
>> or removes device. Also frees allocated resources and info.
>>
>> Signed-off-by: Chaitra P B 
>> Signed-off-by: Suganath Prabu S 
>> ---
>>  drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
>>  drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 
>> +++---
>>  2 files changed, 507 insertions(+), 34 deletions(-)
>>
> Have you considered using 'scan_start()/scan_finished()' SCSI midlayer
> callbacks here?
> Seeing that you are enumerating the devices internally already that
> should give you better control about the scanning process.

Yes mpt3sas driver has defined scan_start() & scan_finished()
callbacks function. In the scan_start() callback function driver will
issue Port Enable request to the HBA Firmware to enable the HBA ports,
so the driver can start receiving drives (SAS/SATA/NVMe) discovery
events from firmware and add the discovered drives to the respective
drive type list. (e.g. SAS & SATA drives are added sas_device_list and
NVMe drives are added to pcie_device_list) . Once the driver receives
the Port Enable completion reply message from firmware then in the
scan_finished() callback function driver will register the SAS & SATA
devices added in the sas_device_list to SCSI Transport Layer by using
sas_rphy_add() API and NVMe devices added in the pcie_device_list are
directly added to SML by using scsi_add_device() API with channel
number set to two.

Thanks,
Sreekanth
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes ReineckeTeamlead Storage & Networking
> h...@suse.de   +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)

Re: [PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-08-03 Thread Sreekanth Reddy 
On Thu, Aug 3, 2017 at 11:57 AM, Hannes Reinecke  wrote:
> On 07/14/2017 03:22 PM, Suganath Prabu S wrote:
>> 1) Added support for probing pcie device and adding NVMe drives to
>> SML and driver's internal list pcie_device_list.
>>
>> 2) Added support for determing NVMe as boot device.
>>
>> 3) Added nvme device support for call back functions scan_finished
>> target_alloc,slave_alloc,target destroy and slave destroy.
>>
>>  a) During scan, pcie devices are probed and added to SML to drivers
>> internal list.
>>
>>  b) target_alloc & slave alloc API's allocates resources for
>> (MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
>> information like handle, target_id etc.
>>
>>  c) slave_destroy & target_destroy are called when driver unregisters
>> or removes device. Also frees allocated resources and info.
>>
>> Signed-off-by: Chaitra P B 
>> Signed-off-by: Suganath Prabu S 
>> ---
>>  drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
>>  drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 
>> +++---
>>  2 files changed, 507 insertions(+), 34 deletions(-)
>>
> Have you considered using 'scan_start()/scan_finished()' SCSI midlayer
> callbacks here?
> Seeing that you are enumerating the devices internally already that
> should give you better control about the scanning process.

Yes mpt3sas driver has defined scan_start() & scan_finished()
callbacks function. In the scan_start() callback function driver will
issue Port Enable request to the HBA Firmware to enable the HBA ports,
so the driver can start receiving drives (SAS/SATA/NVMe) discovery
events from firmware and add the discovered drives to the respective
drive type list. (e.g. SAS & SATA drives are added sas_device_list and
NVMe drives are added to pcie_device_list) . Once the driver receives
the Port Enable completion reply message from firmware then in the
scan_finished() callback function driver will register the SAS & SATA
devices added in the sas_device_list to SCSI Transport Layer by using
sas_rphy_add() API and NVMe devices added in the pcie_device_list are
directly added to SML by using scsi_add_device() API with channel
number set to two.

Thanks,
Sreekanth
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes ReineckeTeamlead Storage & Networking
> h...@suse.de   +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)

Re: [PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-08-03 Thread Hannes Reinecke 
On 07/14/2017 03:22 PM, Suganath Prabu S wrote:
> 1) Added support for probing pcie device and adding NVMe drives to
> SML and driver's internal list pcie_device_list.
> 
> 2) Added support for determing NVMe as boot device.
> 
> 3) Added nvme device support for call back functions scan_finished
> target_alloc,slave_alloc,target destroy and slave destroy.
> 
>  a) During scan, pcie devices are probed and added to SML to drivers
> internal list.
> 
>  b) target_alloc & slave alloc API's allocates resources for
> (MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
> information like handle, target_id etc.
> 
>  c) slave_destroy & target_destroy are called when driver unregisters
> or removes device. Also frees allocated resources and info.
> 
> Signed-off-by: Chaitra P B 
> Signed-off-by: Suganath Prabu S 
> ---
>  drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
>  drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 
> +++---
>  2 files changed, 507 insertions(+), 34 deletions(-)
> 
Have you considered using 'scan_start()/scan_finished()' SCSI midlayer
callbacks here?
Seeing that you are enumerating the devices internally already that
should give you better control about the scanning process.

Cheers,

Hannes
-- 
Dr. Hannes ReineckeTeamlead Storage & Networking
h...@suse.de   +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

Re: [PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-08-03 Thread Hannes Reinecke 
On 07/14/2017 03:22 PM, Suganath Prabu S wrote:
> 1) Added support for probing pcie device and adding NVMe drives to
> SML and driver's internal list pcie_device_list.
> 
> 2) Added support for determing NVMe as boot device.
> 
> 3) Added nvme device support for call back functions scan_finished
> target_alloc,slave_alloc,target destroy and slave destroy.
> 
>  a) During scan, pcie devices are probed and added to SML to drivers
> internal list.
> 
>  b) target_alloc & slave alloc API's allocates resources for
> (MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
> information like handle, target_id etc.
> 
>  c) slave_destroy & target_destroy are called when driver unregisters
> or removes device. Also frees allocated resources and info.
> 
> Signed-off-by: Chaitra P B 
> Signed-off-by: Suganath Prabu S 
> ---
>  drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
>  drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 
> +++---
>  2 files changed, 507 insertions(+), 34 deletions(-)
> 
Have you considered using 'scan_start()/scan_finished()' SCSI midlayer
callbacks here?
Seeing that you are enumerating the devices internally already that
should give you better control about the scanning process.

Cheers,

Hannes
-- 
Dr. Hannes ReineckeTeamlead Storage & Networking
h...@suse.de   +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)

[PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-07-14 Thread Suganath Prabu S 
1) Added support for probing pcie device and adding NVMe drives to
SML and driver's internal list pcie_device_list.

2) Added support for determing NVMe as boot device.

3) Added nvme device support for call back functions scan_finished
target_alloc,slave_alloc,target destroy and slave destroy.

 a) During scan, pcie devices are probed and added to SML to drivers
internal list.

 b) target_alloc & slave alloc API's allocates resources for
(MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
information like handle, target_id etc.

 c) slave_destroy & target_destroy are called when driver unregisters
or removes device. Also frees allocated resources and info.

Signed-off-by: Chaitra P B 
Signed-off-by: Suganath Prabu S 
---
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
 drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 +++---
 2 files changed, 507 insertions(+), 34 deletions(-)

diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h 
b/drivers/scsi/mpt3sas/mpt3sas_base.h
index 099ab4c..c522057 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -159,6 +159,7 @@
 #define MPT_TARGET_FLAGS_VOLUME0x02
 #define MPT_TARGET_FLAGS_DELETED   0x04
 #define MPT_TARGET_FASTPATH_IO 0x08
+#define MPT_TARGET_FLAGS_PCIE_DEVICE   0x10
 
 #define SAS2_PCI_DEVICE_B0_REVISION(0x01)
 #define SAS3_PCI_DEVICE_C0_REVISION(0x02)
@@ -357,7 +358,8 @@ struct Mpi2ManufacturingPage11_t {
  * @flags: MPT_TARGET_FLAGS_XXX flags
  * @deleted: target flaged for deletion
  * @tm_busy: target is busy with TM request.
- * @sdev: The sas_device associated with this target
+ * @sas_dev: The sas_device associated with this target
+ * @pcie_dev: The pcie device associated with this target
  */
 struct MPT3SAS_TARGET {
struct scsi_target *starget;
@@ -368,7 +370,8 @@ struct MPT3SAS_TARGET {
u32 flags;
u8  deleted;
u8  tm_busy;
-   struct _sas_device *sdev;
+   struct _sas_device *sas_dev;
+   struct _pcie_device *pcie_dev;
 };
 
 
@@ -508,6 +511,89 @@ static inline void sas_device_put(struct _sas_device *s)
kref_put(>refcount, sas_device_free);
 }
 
+/*
+ * struct _pcie_device - attached PCIe device information
+ * @list: pcie device list
+ * @starget: starget object
+ * @wwid: device WWID
+ * @handle: device handle
+ * @device_info: bitfield provides detailed info about the device
+ * @id: target id
+ * @channel: target channel
+ * @slot: slot number
+ * @port_num: port number
+ * @responding: used in _scsih_pcie_device_mark_responding
+ * @fast_path: fast path feature enable bit
+ * @nvme_mdts: MaximumDataTransferSize from PCIe Device Page 2 for
+ * NVMe device only
+ * @enclosure_handle: enclosure handle
+ * @enclosure_logical_id: enclosure logical identifier
+ * @enclosure_level: The level of device's enclosure from the controller
+ * @connector_name: ASCII value of the Connector's name
+ * @serial_number: pointer of serial number string allocated runtime
+ * @refcount: reference count for deletion
+ */
+struct _pcie_device {
+   struct list_head list;
+   struct scsi_target *starget;
+   u64 wwid;
+   u16 handle;
+   u32 device_info;
+   int id;
+   int channel;
+   u16 slot;
+   u8  port_num;
+   u8  responding;
+   u8  fast_path;
+   u32 nvme_mdts;
+   u16 enclosure_handle;
+   u64 enclosure_logical_id;
+   u8  enclosure_level;
+   u8  connector_name[4];
+   u8  *serial_number;
+   struct kref refcount;
+};
+/**
+ * pcie_device_get - Increment the pcie device reference count
+ *
+ * @p: pcie_device object
+ *
+ * When ever this function called it will increment the
+ * reference count of the pcie device for which this function called.
+ *
+ */
+static inline void pcie_device_get(struct _pcie_device *p)
+{
+   kref_get(>refcount);
+}
+
+/**
+ * pcie_device_free - Release the pcie device object
+ * @r - kref object
+ *
+ * Free's the pcie device object. It will be called when reference count
+ * reaches to zero.
+ */
+static inline void pcie_device_free(struct kref *r)
+{
+   kfree(container_of(r, struct _pcie_device, refcount));
+}
+
+/**
+ * pcie_device_put - Decrement the pcie device reference count
+ *
+ * @p: pcie_device object
+ *
+ * When ever this function called it will decrement the
+ * reference count of the pcie device for which this function called.
+ *
+ * When refernce count reaches to Zero, this will call pcie_device_free to the
+ * pcie_device object.
+ */
+static inline void pcie_device_put(struct _pcie_device *p)
+{
+   kref_put(>refcount, pcie_device_free);
+}
 /**
  * struct _raid_device - raid volume link list
  * @list: sas device list
@@ -556,12 +642,13 @@ struct _raid_device {
 
 /**
  * struct _boot_device - boot device info
-

[PATCH v2 02/13] mpt3sas: Add nvme device support in slave alloc, target alloc and probe

2017-07-14 Thread Suganath Prabu S 
1) Added support for probing pcie device and adding NVMe drives to
SML and driver's internal list pcie_device_list.

2) Added support for determing NVMe as boot device.

3) Added nvme device support for call back functions scan_finished
target_alloc,slave_alloc,target destroy and slave destroy.

 a) During scan, pcie devices are probed and added to SML to drivers
internal list.

 b) target_alloc & slave alloc API's allocates resources for
(MPT3SAS_TARGET & MPT3SAS_DEVICE) private datas and holds
information like handle, target_id etc.

 c) slave_destroy & target_destroy are called when driver unregisters
or removes device. Also frees allocated resources and info.

Signed-off-by: Chaitra P B 
Signed-off-by: Suganath Prabu S 
---
 drivers/scsi/mpt3sas/mpt3sas_base.h  |  110 -
 drivers/scsi/mpt3sas/mpt3sas_scsih.c |  431 +++---
 2 files changed, 507 insertions(+), 34 deletions(-)

diff --git a/drivers/scsi/mpt3sas/mpt3sas_base.h 
b/drivers/scsi/mpt3sas/mpt3sas_base.h
index 099ab4c..c522057 100644
--- a/drivers/scsi/mpt3sas/mpt3sas_base.h
+++ b/drivers/scsi/mpt3sas/mpt3sas_base.h
@@ -159,6 +159,7 @@
 #define MPT_TARGET_FLAGS_VOLUME0x02
 #define MPT_TARGET_FLAGS_DELETED   0x04
 #define MPT_TARGET_FASTPATH_IO 0x08
+#define MPT_TARGET_FLAGS_PCIE_DEVICE   0x10
 
 #define SAS2_PCI_DEVICE_B0_REVISION(0x01)
 #define SAS3_PCI_DEVICE_C0_REVISION(0x02)
@@ -357,7 +358,8 @@ struct Mpi2ManufacturingPage11_t {
  * @flags: MPT_TARGET_FLAGS_XXX flags
  * @deleted: target flaged for deletion
  * @tm_busy: target is busy with TM request.
- * @sdev: The sas_device associated with this target
+ * @sas_dev: The sas_device associated with this target
+ * @pcie_dev: The pcie device associated with this target
  */
 struct MPT3SAS_TARGET {
struct scsi_target *starget;
@@ -368,7 +370,8 @@ struct MPT3SAS_TARGET {
u32 flags;
u8  deleted;
u8  tm_busy;
-   struct _sas_device *sdev;
+   struct _sas_device *sas_dev;
+   struct _pcie_device *pcie_dev;
 };
 
 
@@ -508,6 +511,89 @@ static inline void sas_device_put(struct _sas_device *s)
kref_put(>refcount, sas_device_free);
 }
 
+/*
+ * struct _pcie_device - attached PCIe device information
+ * @list: pcie device list
+ * @starget: starget object
+ * @wwid: device WWID
+ * @handle: device handle
+ * @device_info: bitfield provides detailed info about the device
+ * @id: target id
+ * @channel: target channel
+ * @slot: slot number
+ * @port_num: port number
+ * @responding: used in _scsih_pcie_device_mark_responding
+ * @fast_path: fast path feature enable bit
+ * @nvme_mdts: MaximumDataTransferSize from PCIe Device Page 2 for
+ * NVMe device only
+ * @enclosure_handle: enclosure handle
+ * @enclosure_logical_id: enclosure logical identifier
+ * @enclosure_level: The level of device's enclosure from the controller
+ * @connector_name: ASCII value of the Connector's name
+ * @serial_number: pointer of serial number string allocated runtime
+ * @refcount: reference count for deletion
+ */
+struct _pcie_device {
+   struct list_head list;
+   struct scsi_target *starget;
+   u64 wwid;
+   u16 handle;
+   u32 device_info;
+   int id;
+   int channel;
+   u16 slot;
+   u8  port_num;
+   u8  responding;
+   u8  fast_path;
+   u32 nvme_mdts;
+   u16 enclosure_handle;
+   u64 enclosure_logical_id;
+   u8  enclosure_level;
+   u8  connector_name[4];
+   u8  *serial_number;
+   struct kref refcount;
+};
+/**
+ * pcie_device_get - Increment the pcie device reference count
+ *
+ * @p: pcie_device object
+ *
+ * When ever this function called it will increment the
+ * reference count of the pcie device for which this function called.
+ *
+ */
+static inline void pcie_device_get(struct _pcie_device *p)
+{
+   kref_get(>refcount);
+}
+
+/**
+ * pcie_device_free - Release the pcie device object
+ * @r - kref object
+ *
+ * Free's the pcie device object. It will be called when reference count
+ * reaches to zero.
+ */
+static inline void pcie_device_free(struct kref *r)
+{
+   kfree(container_of(r, struct _pcie_device, refcount));
+}
+
+/**
+ * pcie_device_put - Decrement the pcie device reference count
+ *
+ * @p: pcie_device object
+ *
+ * When ever this function called it will decrement the
+ * reference count of the pcie device for which this function called.
+ *
+ * When refernce count reaches to Zero, this will call pcie_device_free to the
+ * pcie_device object.
+ */
+static inline void pcie_device_put(struct _pcie_device *p)
+{
+   kref_put(>refcount, pcie_device_free);
+}
 /**
  * struct _raid_device - raid volume link list
  * @list: sas device list
@@ -556,12 +642,13 @@ struct _raid_device {
 
 /**
  * struct _boot_device - boot device info
- * @is_raid: flag to indicate whether this is volume
- * @device: