Re: [RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-27 Thread Rustad, Mark D 
> On Feb 27, 2018, at 7:35 AM, David Miller  wrote:
> 
> I don't like these helpers on many different levels.



> So kill off pci_sriov_enable() helper completely, it is unnecessary,
> and rename the disable helper so that it says something meaningful to
> the reader.

Yes. Once pointed out, I completely agree with your comments and wish that I 
had seen those things myself.

> Thanks.

V3 was junk, but your comments apply to V4 as well, so please ignore it.

Thank you for your valuable review.

-- 
Mark Rustad, Networking Division, Intel Corporation

Re: [RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-27 Thread Rustad, Mark D 
> On Feb 27, 2018, at 7:35 AM, David Miller  wrote:
> 
> I don't like these helpers on many different levels.



> So kill off pci_sriov_enable() helper completely, it is unnecessary,
> and rename the disable helper so that it says something meaningful to
> the reader.

Yes. Once pointed out, I completely agree with your comments and wish that I 
had seen those things myself.

> Thanks.

V3 was junk, but your comments apply to V4 as well, so please ignore it.

Thank you for your valuable review.

-- 
Mark Rustad, Networking Division, Intel Corporation

Re: [RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-27 Thread David Miller 
From: Mark Rustad 
Date: Sun, 25 Feb 2018 19:19:11 -0800

> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
> index 677924ae0350..ddd44a9d93ec 100644
> --- a/drivers/pci/iov.c
> +++ b/drivers/pci/iov.c
> @@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)
>   pci_iov_set_numvfs(dev, 0);
>  }
>  
> +/**
> + * pci_sriov_disable - standard helper to disable SR-IOV
> + * @dev:the PCI PF device whose VFs are to be disabled
> + */
> +int pci_sriov_disable(struct pci_dev *dev)
> +{
> + /*
> +  * If vfs are assigned we cannot shut down SR-IOV without causing
> +  * issues, so just leave the hardware available.
> +  */
> + if (pci_vfs_assigned(dev)) {
> + pci_warn(>dev,
> +  "Cannot disable SR-IOV while VFs are assigned - VFs 
> will not be deallocated\n");
> + return -EPERM;
> + }
> + pci_disable_sriov(dev);
> + return 0;
> +}
> +
> +static int pci_sriov_enable(struct pci_dev *dev, int num_vfs)
> +{
> + int rc;
> +
> + if (pci_num_vf(dev))
> + return -EINVAL;
> +
> + rc = pci_enable_sriov(dev, num_vfs);
> + if (rc) {
> + pci_warn(dev, "Failed to enable PCI sriov: %d\n", rc);
> + return rc;
> + }
> + dev_info(dev, "SR-IOV enabled with %d VFs\n", num_vfs);
> + return num_vfs;
> +}

I don't like these helpers on many different levels.

The pci_num_vf() test in pci_sriov_enable() is redundant, the 
pci_enable_sriov() code
path does that check and returns the same exact error code from sriov_enable().

Just call pci_enable_sriov() directly.  The log message adds no value justifying
an entirely new (and confusingly named) helper.  If the log message is useful, 
add
it to pci_enable_sriov().

Speaking of naming, is this stuff confusing or what?  As a programmer
what am I supposed to think when I consider what may be the difference
between two interfaces, the only difference in naming is that two
words are transposed?

pci_enable_sriov()
pci_sriov_enable()

pci_disable_sriov()
pci_sriov_disable()

?!?!?!?!

As per pci_sriov_disable() explicitly, all it does different is check
for vf assignment and return failure.

If you want a little help that does that, name it appropriately.

pci_disable_sriov_if_unassigned()

So kill off pci_sriov_enable() helper completely, it is unnecessary,
and rename the disable helper so that it says something meaningful to
the reader.

Thanks.

Re: [RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-27 Thread David Miller 
From: Mark Rustad 
Date: Sun, 25 Feb 2018 19:19:11 -0800

> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
> index 677924ae0350..ddd44a9d93ec 100644
> --- a/drivers/pci/iov.c
> +++ b/drivers/pci/iov.c
> @@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)
>   pci_iov_set_numvfs(dev, 0);
>  }
>  
> +/**
> + * pci_sriov_disable - standard helper to disable SR-IOV
> + * @dev:the PCI PF device whose VFs are to be disabled
> + */
> +int pci_sriov_disable(struct pci_dev *dev)
> +{
> + /*
> +  * If vfs are assigned we cannot shut down SR-IOV without causing
> +  * issues, so just leave the hardware available.
> +  */
> + if (pci_vfs_assigned(dev)) {
> + pci_warn(>dev,
> +  "Cannot disable SR-IOV while VFs are assigned - VFs 
> will not be deallocated\n");
> + return -EPERM;
> + }
> + pci_disable_sriov(dev);
> + return 0;
> +}
> +
> +static int pci_sriov_enable(struct pci_dev *dev, int num_vfs)
> +{
> + int rc;
> +
> + if (pci_num_vf(dev))
> + return -EINVAL;
> +
> + rc = pci_enable_sriov(dev, num_vfs);
> + if (rc) {
> + pci_warn(dev, "Failed to enable PCI sriov: %d\n", rc);
> + return rc;
> + }
> + dev_info(dev, "SR-IOV enabled with %d VFs\n", num_vfs);
> + return num_vfs;
> +}

I don't like these helpers on many different levels.

The pci_num_vf() test in pci_sriov_enable() is redundant, the 
pci_enable_sriov() code
path does that check and returns the same exact error code from sriov_enable().

Just call pci_enable_sriov() directly.  The log message adds no value justifying
an entirely new (and confusingly named) helper.  If the log message is useful, 
add
it to pci_enable_sriov().

Speaking of naming, is this stuff confusing or what?  As a programmer
what am I supposed to think when I consider what may be the difference
between two interfaces, the only difference in naming is that two
words are transposed?

pci_enable_sriov()
pci_sriov_enable()

pci_disable_sriov()
pci_sriov_disable()

?!?!?!?!

As per pci_sriov_disable() explicitly, all it does different is check
for vf assignment and return failure.

If you want a little help that does that, name it appropriately.

pci_disable_sriov_if_unassigned()

So kill off pci_sriov_enable() helper completely, it is unnecessary,
and rename the disable helper so that it says something meaningful to
the reader.

Thanks.

[RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-25 Thread Mark Rustad 
Hardware-realized virtio-pci devices can implement SR-IOV, so this
patch enables its use. The device in question is an upcoming Intel
NIC that implements both a virtio-net PF and virtio-net VFs. These
are hardware realizations of what has been up to now been a software
interface.

The device in question has the following 4-part PCI IDs:

PF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 15fe
VF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 05fe

The patch needs no check for device ID, because the callback will
never be made for devices that do not assert the capability or
when run on a platform incapable of SR-IOV.

One reason for this patch is because the hardware requires the
vendor ID of a VF to be the same as the vendor ID of the PF that
created it. So it seemed logical to simply have a fully-functioning
virtio-net PF create the VFs. This patch makes that possible.

Signed-off-by: Mark Rustad 
Reviewed-by: Alexander Duyck 
---
Changes in V3:
- Move most code to a new helper in pci/iov.c, pci_sriov_configure
Changes in V2:
- Simplified logic from previous version, removed added driver variable
- Disable SR-IOV on driver removal excapt when VFs are assigned
- Sent as RFC to virtio-dev, linux-pci, netdev, lkml and others
---
 drivers/pci/iov.c  |   48 
 drivers/virtio/virtio_pci_common.c |2 ++
 include/linux/pci.h|   10 
 3 files changed, 60 insertions(+)

diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 677924ae0350..ddd44a9d93ec 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)
pci_iov_set_numvfs(dev, 0);
 }
 
+/**
+ * pci_sriov_disable - standard helper to disable SR-IOV
+ * @dev:the PCI PF device whose VFs are to be disabled
+ */
+int pci_sriov_disable(struct pci_dev *dev)
+{
+   /*
+* If vfs are assigned we cannot shut down SR-IOV without causing
+* issues, so just leave the hardware available.
+*/
+   if (pci_vfs_assigned(dev)) {
+   pci_warn(>dev,
+"Cannot disable SR-IOV while VFs are assigned - VFs 
will not be deallocated\n");
+   return -EPERM;
+   }
+   pci_disable_sriov(dev);
+   return 0;
+}
+
+static int pci_sriov_enable(struct pci_dev *dev, int num_vfs)
+{
+   int rc;
+
+   if (pci_num_vf(dev))
+   return -EINVAL;
+
+   rc = pci_enable_sriov(dev, num_vfs);
+   if (rc) {
+   pci_warn(dev, "Failed to enable PCI sriov: %d\n", rc);
+   return rc;
+   }
+   dev_info(dev, "SR-IOV enabled with %d VFs\n", num_vfs);
+   return num_vfs;
+}
+
+/**
+ * pci_sriov_configure - standard helper to configure SR-IOV
+ * @dev: the PCI PF device that is configuring SR-IOV
+ */
+int pci_sriov_configure(struct pci_dev *dev, int num_vfs)
+{
+   if (num_vfs)
+   return pci_sriov_enable(dev, num_vfs);
+   if (!pci_num_vf(dev))
+   return -EINVAL;
+   return pci_sriov_disable(dev);
+}
+
 static int sriov_init(struct pci_dev *dev, int pos)
 {
int i, bar64;
diff --git a/drivers/virtio/virtio_pci_common.c 
b/drivers/virtio/virtio_pci_common.c
index 48d4d1cf1cb6..37e353c4f8b4 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -584,6 +584,7 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)
else
virtio_pci_modern_remove(vp_dev);
 
+   pci_sriov_disable(pci_dev);
pci_disable_device(pci_dev);
put_device(dev);
 }
@@ -596,6 +597,7 @@ static struct pci_driver virtio_pci_driver = {
 #ifdef CONFIG_PM_SLEEP
.driver.pm  = _pci_pm_ops,
 #endif
+   .sriov_configure = pci_sriov_configure,
 };
 
 module_pci_driver(virtio_pci_driver);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 024a1beda008..ef6b359afefd 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1947,6 +1947,8 @@ int pci_iov_virtfn_devfn(struct pci_dev *dev, int id);
 
 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
 void pci_disable_sriov(struct pci_dev *dev);
+int pci_sriov_disable(struct pci_dev *dev);
+int pci_sriov_configure(struct pci_dev *dev, int num_vfs);
 int pci_iov_add_virtfn(struct pci_dev *dev, int id);
 void pci_iov_remove_virtfn(struct pci_dev *dev, int id);
 int pci_num_vf(struct pci_dev *dev);
@@ -1973,6 +1975,14 @@ static inline int pci_iov_add_virtfn(struct pci_dev 
*dev, int id)
 static inline void pci_iov_remove_virtfn(struct pci_dev *dev,
 int id) { }
 static inline void pci_disable_sriov(struct pci_dev *dev) { }
+static inline int pci_sriov_disable(struct pci_dev *dev)
+{
+   return -ENODEV;
+}
+static inline int pci_sriov_configure(struct pci_dev *dev, int num_vfs)
+{
+   return -ENODEV;
+}
 static inline int pci_num_vf(struct

[RFC PATCH V3] virtio_pci: Add SR-IOV support

2018-02-25 Thread Mark Rustad 
Hardware-realized virtio-pci devices can implement SR-IOV, so this
patch enables its use. The device in question is an upcoming Intel
NIC that implements both a virtio-net PF and virtio-net VFs. These
are hardware realizations of what has been up to now been a software
interface.

The device in question has the following 4-part PCI IDs:

PF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 15fe
VF: device: 1af4 vendor: 1041 subvendor: 8086 subdevice: 05fe

The patch needs no check for device ID, because the callback will
never be made for devices that do not assert the capability or
when run on a platform incapable of SR-IOV.

One reason for this patch is because the hardware requires the
vendor ID of a VF to be the same as the vendor ID of the PF that
created it. So it seemed logical to simply have a fully-functioning
virtio-net PF create the VFs. This patch makes that possible.

Signed-off-by: Mark Rustad 
Reviewed-by: Alexander Duyck 
---
Changes in V3:
- Move most code to a new helper in pci/iov.c, pci_sriov_configure
Changes in V2:
- Simplified logic from previous version, removed added driver variable
- Disable SR-IOV on driver removal excapt when VFs are assigned
- Sent as RFC to virtio-dev, linux-pci, netdev, lkml and others
---
 drivers/pci/iov.c  |   48 
 drivers/virtio/virtio_pci_common.c |2 ++
 include/linux/pci.h|   10 
 3 files changed, 60 insertions(+)

diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index 677924ae0350..ddd44a9d93ec 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -367,6 +367,54 @@ static void sriov_disable(struct pci_dev *dev)
pci_iov_set_numvfs(dev, 0);
 }
 
+/**
+ * pci_sriov_disable - standard helper to disable SR-IOV
+ * @dev:the PCI PF device whose VFs are to be disabled
+ */
+int pci_sriov_disable(struct pci_dev *dev)
+{
+   /*
+* If vfs are assigned we cannot shut down SR-IOV without causing
+* issues, so just leave the hardware available.
+*/
+   if (pci_vfs_assigned(dev)) {
+   pci_warn(>dev,
+"Cannot disable SR-IOV while VFs are assigned - VFs 
will not be deallocated\n");
+   return -EPERM;
+   }
+   pci_disable_sriov(dev);
+   return 0;
+}
+
+static int pci_sriov_enable(struct pci_dev *dev, int num_vfs)
+{
+   int rc;
+
+   if (pci_num_vf(dev))
+   return -EINVAL;
+
+   rc = pci_enable_sriov(dev, num_vfs);
+   if (rc) {
+   pci_warn(dev, "Failed to enable PCI sriov: %d\n", rc);
+   return rc;
+   }
+   dev_info(dev, "SR-IOV enabled with %d VFs\n", num_vfs);
+   return num_vfs;
+}
+
+/**
+ * pci_sriov_configure - standard helper to configure SR-IOV
+ * @dev: the PCI PF device that is configuring SR-IOV
+ */
+int pci_sriov_configure(struct pci_dev *dev, int num_vfs)
+{
+   if (num_vfs)
+   return pci_sriov_enable(dev, num_vfs);
+   if (!pci_num_vf(dev))
+   return -EINVAL;
+   return pci_sriov_disable(dev);
+}
+
 static int sriov_init(struct pci_dev *dev, int pos)
 {
int i, bar64;
diff --git a/drivers/virtio/virtio_pci_common.c 
b/drivers/virtio/virtio_pci_common.c
index 48d4d1cf1cb6..37e353c4f8b4 100644
--- a/drivers/virtio/virtio_pci_common.c
+++ b/drivers/virtio/virtio_pci_common.c
@@ -584,6 +584,7 @@ static void virtio_pci_remove(struct pci_dev *pci_dev)
else
virtio_pci_modern_remove(vp_dev);
 
+   pci_sriov_disable(pci_dev);
pci_disable_device(pci_dev);
put_device(dev);
 }
@@ -596,6 +597,7 @@ static struct pci_driver virtio_pci_driver = {
 #ifdef CONFIG_PM_SLEEP
.driver.pm  = _pci_pm_ops,
 #endif
+   .sriov_configure = pci_sriov_configure,
 };
 
 module_pci_driver(virtio_pci_driver);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 024a1beda008..ef6b359afefd 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1947,6 +1947,8 @@ int pci_iov_virtfn_devfn(struct pci_dev *dev, int id);
 
 int pci_enable_sriov(struct pci_dev *dev, int nr_virtfn);
 void pci_disable_sriov(struct pci_dev *dev);
+int pci_sriov_disable(struct pci_dev *dev);
+int pci_sriov_configure(struct pci_dev *dev, int num_vfs);
 int pci_iov_add_virtfn(struct pci_dev *dev, int id);
 void pci_iov_remove_virtfn(struct pci_dev *dev, int id);
 int pci_num_vf(struct pci_dev *dev);
@@ -1973,6 +1975,14 @@ static inline int pci_iov_add_virtfn(struct pci_dev 
*dev, int id)
 static inline void pci_iov_remove_virtfn(struct pci_dev *dev,
 int id) { }
 static inline void pci_disable_sriov(struct pci_dev *dev) { }
+static inline int pci_sriov_disable(struct pci_dev *dev)
+{
+   return -ENODEV;
+}
+static inline int pci_sriov_configure(struct pci_dev *dev, int num_vfs)
+{
+   return -ENODEV;
+}
 static inline int pci_num_vf(struct pci_dev *dev) { return 0; }
 static inline int