Re: [PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors

2016-10-15 Thread Hannes Reinecke 

On 10/15/2016 10:47 AM, Christoph Hellwig wrote:

Which cleans up a lot of the MSI-X handling, and allows us to use the
PCI IRQ layer provided vector mapping, which we can then expose to blk-mq.

Signed-off-by: Christoph Hellwig 
---
 drivers/scsi/smartpqi/smartpqi.h  |   2 -
 drivers/scsi/smartpqi/smartpqi_init.c | 102 +++---
 2 files changed, 32 insertions(+), 72 deletions(-)


Reviewed-by: Hannes Reinecke 

Cheers,

Hannes
--
Dr. Hannes Reinecke   zSeries & Storage
h...@suse.de  +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] scsi: allow LLDDs to expose the queue mapping to blk-mq

2016-10-15 Thread Hannes Reinecke 

On 10/15/2016 10:47 AM, Christoph Hellwig wrote:

Just hand through the blk-mq map_queues method in the host template.

Signed-off-by: Christoph Hellwig 
---
 block/blk-mq-cpumap.c|  1 +
 drivers/scsi/scsi_lib.c  | 10 ++
 include/scsi/scsi_host.h |  8 
 3 files changed, 19 insertions(+)

diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
index 19b1d9c..8e61e86 100644
--- a/block/blk-mq-cpumap.c
+++ b/block/blk-mq-cpumap.c
@@ -87,6 +87,7 @@ int blk_mq_map_queues(struct blk_mq_tag_set *set)
free_cpumask_var(cpus);
return 0;
 }
+EXPORT_SYMBOL_GPL(blk_mq_map_queues);

 /*
  * We have no quick way of doing reverse lookups. This is only used at
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 2cca9cf..f23ec24 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1990,6 +1990,15 @@ static void scsi_exit_request(void *data, struct request 
*rq,
kfree(cmd->sense_buffer);
 }

+static int scsi_map_queues(struct blk_mq_tag_set *set)
+{
+   struct Scsi_Host *shost = container_of(set, struct Scsi_Host, tag_set);
+
+   if (shost->hostt->map_queues)
+   return shost->hostt->map_queues(shost);
+   return blk_mq_map_queues(set);
+}
+
 static u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
struct device *host_dev;
@@ -2082,6 +2091,7 @@ static struct blk_mq_ops scsi_mq_ops = {
.timeout= scsi_timeout,
.init_request   = scsi_init_request,
.exit_request   = scsi_exit_request,
+   .map_queues = scsi_map_queues,
 };

 struct request_queue *scsi_mq_alloc_queue(struct scsi_device *sdev)
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 7e4cd53..36680f1 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -278,6 +278,14 @@ struct scsi_host_template {
int (* change_queue_depth)(struct scsi_device *, int);

/*
+* This functions lets the driver expose the queue mapping
+* to the block layer.
+*
+* Status: OPTIONAL
+*/
+   int (* map_queues)(struct Scsi_Host *shost);
+
+   /*
 * This function determines the BIOS parameters for a given
 * harddisk.  These tend to be numbers that are made up by
 * the host adapter.  Parameters:


Reviewed-by: Hannes Reinecke 

Cheers,

Hannes
--
Dr. Hannes Reinecke   zSeries & Storage
h...@suse.de  +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: expose queue the queue mapping for SCSI drivers

2016-10-15 Thread Hannes Reinecke 

On 10/15/2016 10:47 AM, Christoph Hellwig wrote:

In 4.9 I've added support in the interrupt layer to automatically
assign the interrupt affinity at interrupt allocation time, and
expose that information to blk-mq.

This series extents that so that SCSI driver can pass on the information
as well.  The SCSI part is fairly trivial, although we need to also
export the default queue mapping function in blk-mq to keep things simple.

I've also converted over the smartpqi driver as an example as it's the
easiest of the multiqueue SCSI drivers to convert.  I don't actually
have smartpqi hardware, so that part will need additionaly testing.

(I've tested it with virtio_scsi, but that requires a lot of virtio
 changes not ready to post yet)


Very nice.
I've recently looked into exposing the multiqueue settings for 
fcoe/ixgbe, and figured that the SCSI part was indeed missing.

Thanks for doing this.

Cheers,

Hannes
--
Dr. Hannes Reinecke   zSeries & Storage
h...@suse.de  +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/3] blk-mq: export blk_mq_map_queues

2016-10-15 Thread Hannes Reinecke 

On 10/15/2016 10:47 AM, Christoph Hellwig wrote:

This will allow SCSI to have a single blk_mq_ops structure that either
lets the LLDD map the queues to PCIe MSIx vectors or use the default.

Signed-off-by: Christoph Hellwig 
---
 block/blk-mq.h | 1 -
 include/linux/blk-mq.h | 1 +
 2 files changed, 1 insertion(+), 1 deletion(-)

diff --git a/block/blk-mq.h b/block/blk-mq.h
index e5d2524..5347f01 100644
--- a/block/blk-mq.h
+++ b/block/blk-mq.h
@@ -38,7 +38,6 @@ void blk_mq_disable_hotplug(void);
 /*
  * CPU -> queue mappings
  */
-int blk_mq_map_queues(struct blk_mq_tag_set *set);
 extern int blk_mq_hw_queue_to_node(unsigned int *map, unsigned int);

 static inline struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *q,
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index 535ab2e..6c0fb25 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -237,6 +237,7 @@ void blk_mq_unfreeze_queue(struct request_queue *q);
 void blk_mq_freeze_queue_start(struct request_queue *q);
 int blk_mq_reinit_tagset(struct blk_mq_tag_set *set);

+int blk_mq_map_queues(struct blk_mq_tag_set *set);
 void blk_mq_update_nr_hw_queues(struct blk_mq_tag_set *set, int nr_hw_queues);

 /*


Reviewed-by: Hannes Reinecke 

Cheers,

Hannes
--
Dr. Hannes Reinecke   zSeries & Storage
h...@suse.de  +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg)
--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

expose queue the queue mapping for SCSI drivers

2016-10-15 Thread Christoph Hellwig 
In 4.9 I've added support in the interrupt layer to automatically
assign the interrupt affinity at interrupt allocation time, and
expose that information to blk-mq.

This series extents that so that SCSI driver can pass on the information
as well.  The SCSI part is fairly trivial, although we need to also
export the default queue mapping function in blk-mq to keep things simple.

I've also converted over the smartpqi driver as an example as it's the
easiest of the multiqueue SCSI drivers to convert.  I don't actually
have smartpqi hardware, so that part will need additionaly testing.

(I've tested it with virtio_scsi, but that requires a lot of virtio
 changes not ready to post yet)

--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/3] scsi: allow LLDDs to expose the queue mapping to blk-mq

2016-10-15 Thread Christoph Hellwig 
Just hand through the blk-mq map_queues method in the host template.

Signed-off-by: Christoph Hellwig 
---
 block/blk-mq-cpumap.c|  1 +
 drivers/scsi/scsi_lib.c  | 10 ++
 include/scsi/scsi_host.h |  8 
 3 files changed, 19 insertions(+)

diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
index 19b1d9c..8e61e86 100644
--- a/block/blk-mq-cpumap.c
+++ b/block/blk-mq-cpumap.c
@@ -87,6 +87,7 @@ int blk_mq_map_queues(struct blk_mq_tag_set *set)
free_cpumask_var(cpus);
return 0;
 }
+EXPORT_SYMBOL_GPL(blk_mq_map_queues);
 
 /*
  * We have no quick way of doing reverse lookups. This is only used at
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 2cca9cf..f23ec24 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1990,6 +1990,15 @@ static void scsi_exit_request(void *data, struct request 
*rq,
kfree(cmd->sense_buffer);
 }
 
+static int scsi_map_queues(struct blk_mq_tag_set *set)
+{
+   struct Scsi_Host *shost = container_of(set, struct Scsi_Host, tag_set);
+
+   if (shost->hostt->map_queues)
+   return shost->hostt->map_queues(shost);
+   return blk_mq_map_queues(set);
+}
+
 static u64 scsi_calculate_bounce_limit(struct Scsi_Host *shost)
 {
struct device *host_dev;
@@ -2082,6 +2091,7 @@ static struct blk_mq_ops scsi_mq_ops = {
.timeout= scsi_timeout,
.init_request   = scsi_init_request,
.exit_request   = scsi_exit_request,
+   .map_queues = scsi_map_queues,
 };
 
 struct request_queue *scsi_mq_alloc_queue(struct scsi_device *sdev)
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 7e4cd53..36680f1 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -278,6 +278,14 @@ struct scsi_host_template {
int (* change_queue_depth)(struct scsi_device *, int);
 
/*
+* This functions lets the driver expose the queue mapping
+* to the block layer.
+*
+* Status: OPTIONAL
+*/
+   int (* map_queues)(struct Scsi_Host *shost);
+
+   /*
 * This function determines the BIOS parameters for a given
 * harddisk.  These tend to be numbers that are made up by
 * the host adapter.  Parameters:
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-block" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 3/3] smartpqi: switch to pci_alloc_irq_vectors

2016-10-15 Thread Christoph Hellwig 
Which cleans up a lot of the MSI-X handling, and allows us to use the
PCI IRQ layer provided vector mapping, which we can then expose to blk-mq.

Signed-off-by: Christoph Hellwig 
---
 drivers/scsi/smartpqi/smartpqi.h  |   2 -
 drivers/scsi/smartpqi/smartpqi_init.c | 102 +++---
 2 files changed, 32 insertions(+), 72 deletions(-)

diff --git a/drivers/scsi/smartpqi/smartpqi.h b/drivers/scsi/smartpqi/smartpqi.h
index 07b6444..b673825 100644
--- a/drivers/scsi/smartpqi/smartpqi.h
+++ b/drivers/scsi/smartpqi/smartpqi.h
@@ -929,8 +929,6 @@ struct pqi_ctrl_info {
int max_msix_vectors;
int num_msix_vectors_enabled;
int num_msix_vectors_initialized;
-   u32 msix_vectors[PQI_MAX_MSIX_VECTORS];
-   void*intr_data[PQI_MAX_MSIX_VECTORS];
int event_irq;
struct Scsi_Host *scsi_host;
 
diff --git a/drivers/scsi/smartpqi/smartpqi_init.c 
b/drivers/scsi/smartpqi/smartpqi_init.c
index a535b26..8702d9c 100644
--- a/drivers/scsi/smartpqi/smartpqi_init.c
+++ b/drivers/scsi/smartpqi/smartpqi_init.c
@@ -25,6 +25,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -2887,19 +2888,19 @@ static irqreturn_t pqi_irq_handler(int irq, void *data)
 
 static int pqi_request_irqs(struct pqi_ctrl_info *ctrl_info)
 {
+   struct pci_dev *pdev = ctrl_info->pci_dev;
int i;
int rc;
 
-   ctrl_info->event_irq = ctrl_info->msix_vectors[0];
+   ctrl_info->event_irq = pci_irq_vector(pdev, 0);
 
for (i = 0; i < ctrl_info->num_msix_vectors_enabled; i++) {
-   rc = request_irq(ctrl_info->msix_vectors[i],
-   pqi_irq_handler, 0,
-   DRIVER_NAME_SHORT, ctrl_info->intr_data[i]);
+   rc = request_irq(pci_irq_vector(pdev, i), pqi_irq_handler, 0,
+   DRIVER_NAME_SHORT, _info->queue_groups[i]);
if (rc) {
-   dev_err(_info->pci_dev->dev,
+   dev_err(>dev,
"irq %u init failed with error %d\n",
-   ctrl_info->msix_vectors[i], rc);
+   pci_irq_vector(pdev, i), rc);
return rc;
}
ctrl_info->num_msix_vectors_initialized++;
@@ -2908,72 +2909,23 @@ static int pqi_request_irqs(struct pqi_ctrl_info 
*ctrl_info)
return 0;
 }
 
-static void pqi_free_irqs(struct pqi_ctrl_info *ctrl_info)
-{
-   int i;
-
-   for (i = 0; i < ctrl_info->num_msix_vectors_initialized; i++)
-   free_irq(ctrl_info->msix_vectors[i],
-   ctrl_info->intr_data[i]);
-}
-
 static int pqi_enable_msix_interrupts(struct pqi_ctrl_info *ctrl_info)
 {
-   unsigned int i;
-   int max_vectors;
-   int num_vectors_enabled;
-   struct msix_entry msix_entries[PQI_MAX_MSIX_VECTORS];
-
-   max_vectors = ctrl_info->num_queue_groups;
-
-   for (i = 0; i < max_vectors; i++)
-   msix_entries[i].entry = i;
-
-   num_vectors_enabled = pci_enable_msix_range(ctrl_info->pci_dev,
-   msix_entries, PQI_MIN_MSIX_VECTORS, max_vectors);
+   int ret;
 
-   if (num_vectors_enabled < 0) {
+   ret = pci_alloc_irq_vectors(ctrl_info->pci_dev,
+   PQI_MIN_MSIX_VECTORS, ctrl_info->num_queue_groups,
+   PCI_IRQ_MSIX | PCI_IRQ_AFFINITY);
+   if (ret < 0) {
dev_err(_info->pci_dev->dev,
-   "MSI-X init failed with error %d\n",
-   num_vectors_enabled);
-   return num_vectors_enabled;
-   }
-
-   ctrl_info->num_msix_vectors_enabled = num_vectors_enabled;
-   for (i = 0; i < num_vectors_enabled; i++) {
-   ctrl_info->msix_vectors[i] = msix_entries[i].vector;
-   ctrl_info->intr_data[i] = _info->queue_groups[i];
+   "MSI-X init failed with error %d\n", ret);
+   return ret;
}
 
+   ctrl_info->num_msix_vectors_enabled = ret;
return 0;
 }
 
-static void pqi_irq_set_affinity_hint(struct pqi_ctrl_info *ctrl_info)
-{
-   int i;
-   int rc;
-   int cpu;
-
-   cpu = cpumask_first(cpu_online_mask);
-   for (i = 0; i < ctrl_info->num_msix_vectors_initialized; i++) {
-   rc = irq_set_affinity_hint(ctrl_info->msix_vectors[i],
-   get_cpu_mask(cpu));
-   if (rc)
-   dev_err(_info->pci_dev->dev,
-   "error %d setting affinity hint for irq vector 
%u\n",
-   rc, ctrl_info->msix_vectors[i]);
-   cpu = cpumask_next(cpu, cpu_online_mask);
-   }
-}
-
-static void pqi_irq_unset_affinity_hint(struct pqi_ctrl_info *ctrl_info)
-{
-   int i;
-
-   for (i = 0; i <