Re: [PATCH] misc: ibmvsm: Fix wrong assignment of return code
On 8/7/18 7:28 AM, Michael Ellerman wrote:
> "Bryant G. Ly" writes:
>
>> From: "Bryant G. Ly"
>>
>> Currently the assignment is flipped and rc is always 0.
> If you'd left rc uninitialised at the start of the function the compiler
> would have caught it for you.
>
> And what is the consequence of the bug? Nothing, complete system crash,
> subtle data corruption?
The consequence would be that if the CRQ Registration failed the first time
due to not enough resources, it would never try to reset and try again.
If it fails due to any other error then it would just fail the sending of the
crq init message, thus it would just wait for the client to init, which would
never happen.
We would also have a memory leak since in the error case DMA would never get
un-mapped and the message queue never gets freed.
>
> Also this should be tagged:
>
> Fixes: 0eca353e7ae7 ("misc: IBM Virtual Management Channel Driver (VMC)")
>
> cheers
>
Yep, sorry I forgot to add the Fixes:..
-Bryant
[PATCH] misc: ibmvsm: Fix wrong assignment of return code
From: "Bryant G. Ly" Currently the assignment is flipped and rc is always 0. Signed-off-by: Bryant G. Ly Reviewed-by: Bradley Warrum --- drivers/misc/ibmvmc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/misc/ibmvmc.c b/drivers/misc/ibmvmc.c index 8f82bb9..b8aaa68 100644 --- a/drivers/misc/ibmvmc.c +++ b/drivers/misc/ibmvmc.c @@ -2131,7 +2131,7 @@ static int ibmvmc_init_crq_queue(struct crq_server_adapter *adapter) retrc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, queue->msg_token, PAGE_SIZE); - retrc = rc; + rc = retrc; if (rc == H_RESOURCE) rc = ibmvmc_reset_crq_queue(adapter); -- 2.7.2
Re: [PATCH v3 1/1] powerpc/pseries: fix EEH recovery of some IOV devices
On 7/29/18 8:59 PM, Sam Bobroff wrote: > EEH recovery currently fails on pSeries for some IOV capable PCI > devices, if CONFIG_PCI_IOV is on and the hypervisor doesn't provide > certain device tree properties for the device. (Found on an IOV > capable device using the ipr driver.) > > Recovery fails in pci_enable_resources() at the check on r->parent, > because r->flags is set and r->parent is not. This state is due to > sriov_init() setting the start, end and flags members of the IOV BARs > but the parent not being set later in > pseries_pci_fixup_iov_resources(), because the > "ibm,open-sriov-vf-bar-info" property is missing. > > Correct this by zeroing the resource flags for IOV BARs when they > can't be configured (this is the same method used by sriov_init() and > __pci_read_base()). > > VFs cleared this way can't be enabled later, because that requires > another device tree property, "ibm,number-of-configurable-vfs" as well > as support for the RTAS function "ibm_map_pes". These are all part of > hypervisor support for IOV and it seems unlikely that a hypervisor > would ever partially, but not fully, support it. (None are currently > provided by QEMU/KVM.) > > Signed-off-by: Sam Bobroff > --- > Hi, > > This is a fix to allow EEH recovery to succeed in a specific situation, > which I've tried to explain in the commit message. > > As with the RFC version, the IOV BARs are disabled by setting the resource > flags to 0 but the other fields are now left as-is because that is what is > done > elsewhere (see sriov_init() and __pci_read_base()). > > I've also examined the concern raised by Bjorn Helgaas, that VFs could be > enabled later after the BARs are disabled, and it already seems safe: enabling > VFs (on pseries) depends on another device tree property, > "ibm,number-of-configurable-vfs" as well as support for the RTAS function > "ibm_map_pes". Since these are all part of the hypervisor's support for IOV it > seems unlikely that we would ever see some of them but not all. (None are > currently provided by QEMU/KVM.) (Additionally, the ipr driver on which the > EEH > recovery failure was discovered doesn't even seem to have SR-IOV support so it > certainly can't enable VFs.) > > Cheers, > Sam. > > Patch set v3: > Patch 1/1: powerpc/pseries: fix EEH recovery of some IOV devices > * Moved some useful information from the cover letter to the commit log. > > Patch set v2: > Patch 1/1: powerpc/pseries: fix EEH recovery of some IOV devices > * Moved the BAR disabling code to a function. > * Also check in pseries_pci_fixup_resources(). > > Patch set v1: > Patch 1/1: powerpc/pseries: fix EEH recovery of IOV devices > > arch/powerpc/platforms/pseries/setup.c | 25 + > 1 file changed, 17 insertions(+), 8 deletions(-) > > diff --git a/arch/powerpc/platforms/pseries/setup.c > b/arch/powerpc/platforms/pseries/setup.c > index b55ad4286dc7..0a9e4243ae1d 100644 > --- a/arch/powerpc/platforms/pseries/setup.c > +++ b/arch/powerpc/platforms/pseries/setup.c > @@ -645,6 +645,15 @@ void of_pci_parse_iov_addrs(struct pci_dev *dev, const > int *indexes) > } > } Reviewed-by: Bryant G. Ly
Re: [PATCH v2 1/1] powerpc/pseries: fix EEH recovery of some IOV devices
On 5/24/18 7:31 PM, Sam Bobroff wrote:
> EEH recovery currently fails on pSeries for some IOV capable PCI
> devices, if CONFIG_PCI_IOV is on and the hypervisor doesn't provide
> certain device tree properties for the device. (Found on an IOV
> capable device using the ipr driver.)
>
> Recovery fails in pci_enable_resources() at the check on r->parent,
> because r->flags is set and r->parent is not. This state is due to
> sriov_init() setting the start, end and flags members of the IOV BARs
> but the parent not being set later in
> pseries_pci_fixup_iov_resources(), because the
> "ibm,open-sriov-vf-bar-info" property is missing.
>
> Correct this by zeroing the resource flags for IOV BARs when they
> can't be configured.
>
> Signed-off-by: Sam Bobroff
> ---
> Hi,
>
> This is a fix to allow EEH recovery to succeed in a specific situation,
> which I've tried to explain in the commit message.
>
> As with the RFC version, the IOV BARs are disabled by setting the resource
> flags to 0 but the other fields are now left as-is because that is what is
> done
> elsewhere (see sriov_init() and __pci_read_base()).
>
> I've also examined the concern raised by Bjorn Helgaas, that VFs could be
> enabled later after the BARs are disabled, and it already seems safe: enabling
> VFs (on pseries) depends on another device tree property,
> "ibm,number-of-configurable-vfs" as well as support for the RTAS function
> "ibm_map_pes". Since these are all part of the hypervisor's support for IOV it
> seems unlikely that we would ever see some of them but not all. (None are
> currently provided by QEMU/KVM.) (Additionally, the ipr driver on which the
> EEH
> recovery failure was discovered doesn't even seem to have SR-IOV support so it
> certainly can't enable VFs.)
>
> Cheers,
> Sam.
> == v1 -> v2: ==
>
> Patch 1/1: powerpc/pseries: fix EEH recovery of some IOV devices
> * Moved the BAR disabling code to a function.
> * Also check in pseries_pci_fixup_resources().
>
> == v1: ==
>
> Patch 1/1: powerpc/pseries: fix EEH recovery of IOV devices
>
> arch/powerpc/platforms/pseries/setup.c | 25 +
> 1 file changed, 17 insertions(+), 8 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/setup.c
> b/arch/powerpc/platforms/pseries/setup.c
> index b55ad4286dc7..0a9e4243ae1d 100644
> --- a/arch/powerpc/platforms/pseries/setup.c
> +++ b/arch/powerpc/platforms/pseries/setup.c
> @@ -645,6 +645,15 @@ void of_pci_parse_iov_addrs(struct pci_dev *dev, const
> int *indexes)
> }
> }
>
> +static void pseries_disable_sriov_resources(struct pci_dev *pdev)
> +{
> + int i;
> +
> + pci_warn(pdev, "No hypervisor support for SR-IOV on this device, IOV
> BARs disabled.\n");
> + for (i = 0; i < PCI_SRIOV_NUM_BARS; i++)
> + pdev->resource[i + PCI_IOV_RESOURCES].flags = 0;
> +}
> +
> static void pseries_pci_fixup_resources(struct pci_dev *pdev)
> {
> const int *indexes;
> @@ -652,10 +661,10 @@ static void pseries_pci_fixup_resources(struct pci_dev
> *pdev)
>
> /*Firmware must support open sriov otherwise dont configure*/
> indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
> - if (!indexes)
> - return;
> - /* Assign the addresses from device tree*/
> - of_pci_set_vf_bar_size(pdev, indexes);
> + if (indexes)
> + of_pci_set_vf_bar_size(pdev, indexes);
> + else
> + pseries_disable_sriov_resources(pdev);
> }
>
> static void pseries_pci_fixup_iov_resources(struct pci_dev *pdev)
> @@ -667,10 +676,10 @@ static void pseries_pci_fixup_iov_resources(struct
> pci_dev *pdev)
> return;
> /*Firmware must support open sriov otherwise dont configure*/
> indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
> - if (!indexes)
> - return;
> - /* Assign the addresses from device tree*/
> - of_pci_parse_iov_addrs(pdev, indexes);
> + if (indexes)
> + of_pci_parse_iov_addrs(pdev, indexes);
> + else
> + pseries_disable_sriov_resources(pdev);
> }
>
> static resource_size_t pseries_pci_iov_resource_alignment(struct pci_dev
> *pdev,
Acked-by: Bryant G. Ly
[PATCH v2 1/1] misc: IBM Virtual Management Channel Driver (VMC)
This driver is a logical device which provides an interface between the hypervisor and a management partition. This interface is like a message passing interface. This management partition is intended to provide an alternative to HMC-based system management. VMC enables the Management LPAR to provide basic logical partition functions: - Logical Partition Configuration - Boot, start, and stop actions for individual partitions - Display of partition status - Management of virtual Ethernet - Management of virtual Storage - Basic system management This driver is to be used for the POWER Virtual Management Channel Virtual Adapter on the PowerPC platform. It provides a character device which allows for both request/response and async message support through the /dev/ibmvmc node. Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> Reviewed-by: Steven Royer <sero...@linux.vnet.ibm.com> Reviewed-by: Adam Reznechek <adrez...@linux.vnet.ibm.com> Reviewed-by: Randy Dunlap <rdun...@infradead.org> Tested-by: Taylor Jakobson <tjak...@us.ibm.com> Tested-by: Brad Warrum <bwar...@us.ibm.com> Cc: Greg Kroah-Hartman <gre...@linuxfoundation.org> Cc: Arnd Bergmann <a...@arndb.de> Cc: Benjamin Herrenschmidt <b...@kernel.crashing.org> Cc: Michael Ellerman <m...@ellerman.id.au> --- Documentation/ioctl/ioctl-number.txt |1 + Documentation/misc-devices/ibmvmc.rst | 226 +++ MAINTAINERS |6 + arch/powerpc/include/asm/hvcall.h |1 + drivers/misc/Kconfig | 14 + drivers/misc/Makefile |1 + drivers/misc/ibmvmc.c | 2418 + drivers/misc/ibmvmc.h | 209 +++ 8 files changed, 2876 insertions(+) create mode 100644 Documentation/misc-devices/ibmvmc.rst create mode 100644 drivers/misc/ibmvmc.c create mode 100644 drivers/misc/ibmvmc.h diff --git a/Documentation/ioctl/ioctl-number.txt b/Documentation/ioctl/ioctl-number.txt index 84bb74d..9851cee 100644 --- a/Documentation/ioctl/ioctl-number.txt +++ b/Documentation/ioctl/ioctl-number.txt @@ -329,6 +329,7 @@ Code Seq#(hex) Include FileComments 0xCA 80-BF uapi/scsi/cxlflash_ioctl.h 0xCB 00-1F CBM serial IEC bus in development: <mailto:michael.kl...@puffin.lb.shuttle.de> +0xCC 00-0F drivers/misc/ibmvmc.hpseries VMC driver 0xCD 01 linux/reiserfs_fs.h 0xCF 02 fs/cifs/ioctl.c 0xDB 00-0F drivers/char/mwave/mwavepub.h diff --git a/Documentation/misc-devices/ibmvmc.rst b/Documentation/misc-devices/ibmvmc.rst new file mode 100644 index 000..46ded79 --- /dev/null +++ b/Documentation/misc-devices/ibmvmc.rst @@ -0,0 +1,226 @@ +.. SPDX-License-Identifier: GPL-2.0+ +== +IBM Virtual Management Channel Kernel Driver (IBMVMC) +== + +:Authors: + Dave Engebretsen <engeb...@us.ibm.com>, + Adam Reznechek <adrez...@linux.vnet.ibm.com>, + Steven Royer <sero...@linux.vnet.ibm.com>, + Bryant G. Ly <bryan...@linux.vnet.ibm.com>, + +Introduction + + +Note: Knowledge of virtualization technology is required to understand +this document. + +A good reference document would be: + +https://openpowerfoundation.org/wp-content/uploads/2016/05/LoPAPR_DRAFT_v11_24March2016_cmt1.pdf + +The Virtual Management Channel (VMC) is a logical device which provides an +interface between the hypervisor and a management partition. This interface +is like a message passing interface. This management partition is intended +to provide an alternative to systems that use a Hardware Management +Console (HMC) - based system management. + +The primary hardware management solution that is developed by IBM relies +on an appliance server named the Hardware Management Console (HMC), +packaged as an external tower or rack-mounted personal computer. In a +Power Systems environment, a single HMC can manage multiple POWER +processor-based systems. + +Management Application +-- + +In the management partition, a management application exists which enables +a system administrator to configure the system’s partitioning +characteristics via a command line interface (CLI) or Representational +State Transfer Application (REST API's). + +The management application runs on a Linux logical partition on a +POWER8 or newer processor-based server that is virtualized by PowerVM. +System configuration, maintenance, and control functions which +traditionally require an HMC can be implemented in the management +application using a combination of HMC to hypervisor interfaces and +existing operating system methods. This tool provides a subset of the +functions implemented by the HMC and enables basic partition configuration. +The set of HMC to hypervisor messages
[PATCH v2 0/1] misc: IBM Virtual Management Channel Driver (VMC)
Steven Royer had previously attempted to upstream this driver two years ago, but never got the chance to address the concerns from Greg Kroah-Hartman. The thread with the initial upstream is: https://lkml.org/lkml/2016/2/16/918 I have addressed the following: Version 1: - Documentation - Use of dev_dbg instead of pr_dbg - Change to misc class - Fixed memory barrier usages - Addressed styling, checkpatch, renaming of functions - General fixes to the driver to make it more inline with existing upstream drivers. Version 2: - Changed Documentation from .rst to .txt and addressed small changes in documentation per request from Randy. - Clarified Documentation based upon Linus Walleij's comments - Fixed kbuild warning in regards to unititialized use of rc Bryant G. Ly (1): misc: IBM Virtual Management Channel Driver (VMC) Documentation/ioctl/ioctl-number.txt |1 + Documentation/misc-devices/ibmvmc.rst | 226 +++ MAINTAINERS |6 + arch/powerpc/include/asm/hvcall.h |1 + drivers/misc/Kconfig | 14 + drivers/misc/Makefile |1 + drivers/misc/ibmvmc.c | 2418 + drivers/misc/ibmvmc.h | 209 +++ 8 files changed, 2876 insertions(+) create mode 100644 Documentation/misc-devices/ibmvmc.rst create mode 100644 drivers/misc/ibmvmc.c create mode 100644 drivers/misc/ibmvmc.h -- 2.7.2
Re: [PATCH v1 1/1] misc: IBM Virtual Management Channel Driver
On 4/23/18 2:53 PM, Greg KH wrote: > On Mon, Apr 23, 2018 at 11:38:18AM -0700, Randy Dunlap wrote: >> On 04/23/18 07:46, Bryant G. Ly wrote: >>> This driver is a logical device which provides an >>> interface between the hypervisor and a management >>> partition. >>> >>> This driver is to be used for the POWER Virtual >>> Management Channel Virtual Adapter on the PowerVM >>> platform. It provides both request/response and >>> async message support through the /dev/ibmvmc node. >>> >>> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> >>> Reviewed-by: Steven Royer <sero...@linux.vnet.ibm.com> >>> Reviewed-by: Adam Reznechek <adrez...@linux.vnet.ibm.com> >>> Tested-by: Taylor Jakobson <tjak...@us.ibm.com> >>> Tested-by: Brad Warrum <bwar...@us.ibm.com> >>> Cc: Greg Kroah-Hartman <gre...@linuxfoundation.org> >>> Cc: Arnd Bergmann <a...@arndb.de> >>> Cc: Benjamin Herrenschmidt <b...@kernel.crashing.org> >>> Cc: Michael Ellerman <m...@ellerman.id.au> >>> --- >>> Documentation/ioctl/ioctl-number.txt |1 + >>> Documentation/misc-devices/ibmvmc.txt | 161 +++ >>> MAINTAINERS |6 + >>> arch/powerpc/include/asm/hvcall.h |1 + >>> drivers/misc/Kconfig | 14 + >>> drivers/misc/Makefile |1 + >>> drivers/misc/ibmvmc.c | 2415 >>> + >>> drivers/misc/ibmvmc.h | 209 +++ >>> 8 files changed, 2808 insertions(+) >>> create mode 100644 Documentation/misc-devices/ibmvmc.txt >>> create mode 100644 drivers/misc/ibmvmc.c >>> create mode 100644 drivers/misc/ibmvmc.h >>> diff --git a/Documentation/misc-devices/ibmvmc.txt >>> b/Documentation/misc-devices/ibmvmc.txt >>> new file mode 100644 >>> index 000..bae1064 >>> --- /dev/null >>> +++ b/Documentation/misc-devices/ibmvmc.txt > Aren't we doing new documentation in .rst format instead of .txt? > > thanks, > > greg k-h > I will convert to .rst and fix the comments from Randy in the next version. Thanks, Bryant
[PATCH v1 1/1] misc: IBM Virtual Management Channel Driver
This driver is a logical device which provides an interface between the hypervisor and a management partition. This driver is to be used for the POWER Virtual Management Channel Virtual Adapter on the PowerVM platform. It provides both request/response and async message support through the /dev/ibmvmc node. Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> Reviewed-by: Steven Royer <sero...@linux.vnet.ibm.com> Reviewed-by: Adam Reznechek <adrez...@linux.vnet.ibm.com> Tested-by: Taylor Jakobson <tjak...@us.ibm.com> Tested-by: Brad Warrum <bwar...@us.ibm.com> Cc: Greg Kroah-Hartman <gre...@linuxfoundation.org> Cc: Arnd Bergmann <a...@arndb.de> Cc: Benjamin Herrenschmidt <b...@kernel.crashing.org> Cc: Michael Ellerman <m...@ellerman.id.au> --- Documentation/ioctl/ioctl-number.txt |1 + Documentation/misc-devices/ibmvmc.txt | 161 +++ MAINTAINERS |6 + arch/powerpc/include/asm/hvcall.h |1 + drivers/misc/Kconfig | 14 + drivers/misc/Makefile |1 + drivers/misc/ibmvmc.c | 2415 + drivers/misc/ibmvmc.h | 209 +++ 8 files changed, 2808 insertions(+) create mode 100644 Documentation/misc-devices/ibmvmc.txt create mode 100644 drivers/misc/ibmvmc.c create mode 100644 drivers/misc/ibmvmc.h diff --git a/Documentation/ioctl/ioctl-number.txt b/Documentation/ioctl/ioctl-number.txt index 84bb74d..9851cee 100644 --- a/Documentation/ioctl/ioctl-number.txt +++ b/Documentation/ioctl/ioctl-number.txt @@ -329,6 +329,7 @@ Code Seq#(hex) Include FileComments 0xCA 80-BF uapi/scsi/cxlflash_ioctl.h 0xCB 00-1F CBM serial IEC bus in development: <mailto:michael.kl...@puffin.lb.shuttle.de> +0xCC 00-0F drivers/misc/ibmvmc.hpseries VMC driver 0xCD 01 linux/reiserfs_fs.h 0xCF 02 fs/cifs/ioctl.c 0xDB 00-0F drivers/char/mwave/mwavepub.h diff --git a/Documentation/misc-devices/ibmvmc.txt b/Documentation/misc-devices/ibmvmc.txt new file mode 100644 index 000..bae1064 --- /dev/null +++ b/Documentation/misc-devices/ibmvmc.txt @@ -0,0 +1,161 @@ +Kernel Driver ibmvmc + + +Authors: + Dave Engebretsen <engeb...@us.ibm.com> + Adam Reznechek <adrez...@linux.vnet.ibm.com> + Steven Royer <sero...@linux.vnet.ibm.com> + Bryant G. Ly <bryan...@linux.vnet.ibm.com> + +Description +=== + +The Virtual Management Channel (VMC) is a logical device which provides an +interface between the hypervisor and a management partition. This +management partition is intended to provide an alternative to HMC-based +system management. In the management partition, a Novalink application +exists which enables a system administrator to configure the system’s +partitioning characteristics via a command line interface (CLI) or +Representational State Transfer Application (REST). You can also manage the +server by using PowerVC or other OpenStack solution. Support for +conventional HMC management of the system may still be provided on a +system; however, when an HMC is attached to the system, the VMC +interface is disabled by the hypervisor. + +NovaLink runs on a Linux logical partition on a POWER8 or newer processor- +based server that is virtualized by PowerVM. System configuration, +maintenance, and control functions which traditionally require an HMC can +be implemented in the Novalink using a combination of HMC to hypervisor +interfaces and existing operating system methods. This tool provides a +subset of the functions implemented by the HMC and enables basic partition +configuration. The set of HMC to hypervisor messages supported by the +Novalink component are passed to the hypervisor over a VMC interface, which +is defined below. + +Virtual Management Channel (VMC) +A logical device, called the virtual management channel (VMC), is defined +for communicating between the Novalink application and the hypervisor. +This device, similar to a VSCSI server device, is presented to a designated +management partition as a virtual device and is only presented when the +system is not HMC managed. +This communication device borrows aspects from both VSCSI and ILLAN devices +and is implemented using the CRQ and the RDMA interfaces. A three-way +handshake is defined that must take place to establish that both the +hypervisor and management partition sides of the channel are running prior +to sending/receiving any of the protocol messages. +This driver also utilizes Transport Event CRQs. CRQ messages that are sent +when the hypervisor detects one of the peer partitions has abnormally +terminated, or one side has called H_FREE_CRQ to close their CRQ. +Two new classes of CRQ messages are introduced for the VMC device. VMC +Administrative messages are used for each partition using th
[PATCH v1 0/1] misc: IBM Virtual Management Channel Driver
Steven Royer had previously attempted to upstream this driver two years ago, but never got the chance to address the concerns from Greg Kroah-Hartman. The thread with the initial upstream is: https://lkml.org/lkml/2016/2/16/918 I have addressed the following: - Documentation - Use of dev_dbg instead of pr_dbg - Change to misc class - Fixed memory barrier usages - Addressed styling, checkpatch, renaming of functions - General fixes to the driver to make it more inline with existing upstream drivers. Bryant G. Ly (1): misc: IBM Virtual Management Channel Driver Documentation/ioctl/ioctl-number.txt |1 + Documentation/misc-devices/ibmvmc.txt | 161 +++ MAINTAINERS |6 + arch/powerpc/include/asm/hvcall.h |1 + drivers/misc/Kconfig |9 + drivers/misc/Makefile |1 + drivers/misc/ibmvmc.c | 2413 + drivers/misc/ibmvmc.h | 215 +++ 8 files changed, 2807 insertions(+) create mode 100644 Documentation/misc-devices/ibmvmc.txt create mode 100644 drivers/misc/ibmvmc.c create mode 100644 drivers/misc/ibmvmc.h -- 2.7.2
[PATCH] tty:hvcs: Move Assignment for hvcsd->port.count
Move the assignment out of hvcsd->port.count from within the
if () block so this patch fixes it. It is bad practice to have
assignments within an if () block.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
---
drivers/tty/hvc/hvcs.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/tty/hvc/hvcs.c b/drivers/tty/hvc/hvcs.c
index 1db1d97..333dc76 100644
--- a/drivers/tty/hvc/hvcs.c
+++ b/drivers/tty/hvc/hvcs.c
@@ -1202,7 +1202,8 @@ static void hvcs_close(struct tty_struct *tty, struct
file *filp)
hvcsd = tty->driver_data;
spin_lock_irqsave(>lock, flags);
- if (--hvcsd->port.count == 0) {
+ --hvcsd->port.count;
+ if (hvcsd->port.count == 0) {
vio_disable_interrupts(hvcsd->vdev);
--
2.7.2
Re: [PATCH] PCI/AER: Move pci_uevent_ers() out of pci.h
On 2/22/18 10:10 PM, Michael Ellerman wrote: > Bjorn Helgaas <helg...@kernel.org> writes: > >> On Thu, Feb 08, 2018 at 09:05:45AM -0600, Bryant G. Ly wrote: >>> On 2/8/18 6:20 AM, Michael Ellerman wrote: >>> >>>> There's no reason pci_uevent_ers() needs to be inline in pci.h, so >>>> move it out to a C file. >>>> >>>> Given it's used by AER the obvious location would be somewhere in >>>> drivers/pci/pcie/aer, but because it's also used by powerpc EEH code >>>> unfortunately that doesn't work in the case where EEH is enabled but >>>> PCIEPORTBUS is not. >>>> >>>> So for now put it in pci-driver.c, next to pci_uevent(), with an >>>> appropriate #ifdef so it's not built if AER and EEH are both disabled. >>>> >>>> While we're moving it also fix up the kernel doc comment for @pdev to >>>> be accurate. >>>> >>>> Reported-by: Linus Torvalds <torva...@linux-foundation.org> >>>> Signed-off-by: Michael Ellerman <m...@ellerman.id.au> >>>> --- >>>> drivers/pci/pci-driver.c | 36 >>>> include/linux/pci.h | 38 +++--- >>>> 2 files changed, 39 insertions(+), 35 deletions(-) >>> Looks good, thanks for fixing it! >>> >>> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> >> This would normally be a "Reviewed-by" unless you actually >> participated in developing the patch, and in that case, your >> "Signed-off-by" would normally be included in the original posting. >> >> What do you intend? I'll be glad to add either. > I wrote the patch, so Bryant meant Reviewed-by or maybe Acked-by. > > cheers > Yes, I mean't, Reviewed-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> Thanks, Bryant
Re: [PATCH] PCI/AER: Add a null check before eeh_ops->notify_resume callback.
On 2/22/18 5:58 AM, Vaibhav Jain wrote:
> This patch puts a NULL check before branching to the address pointed
> to by eeh_ops->notify_resume in eeh_report_resume(). The callback
> is used to notify the arch EEH code that a pci device is back
> online.
>
> For PPC64 presently, only an implementation for pseries platform is
> available and not for powernv. Hence without this patch EEH recovery
> on all non-virtualized hosts is causing a kernel panic when
> CONFIG_PCI_IOV is set. The panic is usually is of the form:
>
> EEH: Notify device driver to resume
> Unable to handle kernel paging request for instruction fetch
> Faulting instruction address: 0x
> Oops: Kernel access of bad area, sig: 11 [#1]
>
> LR eeh_report_resume+0x218/0x220
> Call Trace:
> eeh_report_resume+0x1f0/0x220 (unreliable)
> eeh_pe_dev_traverse+0x98/0x170
> eeh_handle_normal_event+0x3f4/0x650
> eeh_handle_event+0x188/0x380
> eeh_event_handler+0x208/0x210
> kthread+0x168/0x1b0
> ret_from_kernel_thread+0x5c/0xb4
>
> Cc: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
> Fixes: 856e1eb9bdd4("PCI/AER: Add uevents in AER and EEH error/resume")
> Signed-off-by: Vaibhav Jain <vaib...@linux.vnet.ibm.com>
> ---
> arch/powerpc/kernel/eeh_driver.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/kernel/eeh_driver.c
> b/arch/powerpc/kernel/eeh_driver.c
> index beea2182d754..932858a293ea 100644
> --- a/arch/powerpc/kernel/eeh_driver.c
> +++ b/arch/powerpc/kernel/eeh_driver.c
> @@ -384,7 +384,8 @@ static void *eeh_report_resume(void *data, void *userdata)
> eeh_pcid_put(dev);
> pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
> #ifdef CONFIG_PCI_IOV
> - eeh_ops->notify_resume(eeh_dev_to_pdn(edev));
> + if (eeh_ops->notify_resume)
> + eeh_ops->notify_resume(eeh_dev_to_pdn(edev));
> #endif
> return NULL;
> }
A version of this patch already upstreamed.
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?h=fixes=521ca5a9859a870e354d1a6b84a6ff4c07bbceb0
-Bryant
[PATCH] powerpc/eeh: Add conditional check on notify_resume
From: "Juan J. Alvarez" <jjalv...@linux.vnet.ibm.com> EEH structure is not populated with function notify resume when running on systems that do not support it, i.e: BMC. Hence adding a conditional check for NULL for systems that don't add function notify_resume. Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com> Reviewed-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> Tested-by: Carol L. Soto <cls...@us.ibm.com> --- arch/powerpc/kernel/eeh_driver.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c index beea218..0c0b66f 100644 --- a/arch/powerpc/kernel/eeh_driver.c +++ b/arch/powerpc/kernel/eeh_driver.c @@ -384,7 +384,8 @@ static void *eeh_report_resume(void *data, void *userdata) eeh_pcid_put(dev); pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED); #ifdef CONFIG_PCI_IOV - eeh_ops->notify_resume(eeh_dev_to_pdn(edev)); + if (eeh_ops->notify_resume && eeh_dev_to_pdn(edev)) + eeh_ops->notify_resume(eeh_dev_to_pdn(edev)); #endif return NULL; } -- 2.7.2
Re: [PATCH] PCI/AER: Move pci_uevent_ers() out of pci.h
On 2/8/18 6:20 AM, Michael Ellerman wrote: > There's no reason pci_uevent_ers() needs to be inline in pci.h, so > move it out to a C file. > > Given it's used by AER the obvious location would be somewhere in > drivers/pci/pcie/aer, but because it's also used by powerpc EEH code > unfortunately that doesn't work in the case where EEH is enabled but > PCIEPORTBUS is not. > > So for now put it in pci-driver.c, next to pci_uevent(), with an > appropriate #ifdef so it's not built if AER and EEH are both disabled. > > While we're moving it also fix up the kernel doc comment for @pdev to > be accurate. > > Reported-by: Linus Torvalds <torva...@linux-foundation.org> > Signed-off-by: Michael Ellerman <m...@ellerman.id.au> > --- > drivers/pci/pci-driver.c | 36 > include/linux/pci.h | 38 +++--- > 2 files changed, 39 insertions(+), 35 deletions(-) Looks good, thanks for fixing it! Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> -Bryant
Re: [PATCH v4 3/7] platforms/pseries: Set eeh_pe of EEH_PE_VF type
On 1/23/18 7:14 PM, Michael Ellerman wrote:
> "Bryant G. Ly" <bryan...@linux.vnet.ibm.com> writes:
>
>> To correctly use EEH code one has to make
>> sure that the EEH_PE_VF is set for dynamic created
>> VFs. Therefore this patch allocates an eeh_pe of
>> eeh type EEH_PE_VF and associates PE with parent.
>>
>> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
>> Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/include/asm/pci-bridge.h| 5 -
>> arch/powerpc/platforms/pseries/eeh_pseries.c | 17 +
>> 2 files changed, 21 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/include/asm/pci-bridge.h
>> b/arch/powerpc/include/asm/pci-bridge.h
>> index 9f66ddebb799..16d70740a76f 100644
>> --- a/arch/powerpc/include/asm/pci-bridge.h
>> +++ b/arch/powerpc/include/asm/pci-bridge.h
>> @@ -211,7 +211,10 @@ struct pci_dn {
>> unsigned int *pe_num_map; /* PE# for the first VF PE or array */
>> boolm64_single_mode;/* Use M64 BAR in Single Mode */
>> #define IODA_INVALID_M64(-1)
>> -int (*m64_map)[PCI_SRIOV_NUM_BARS];
>> +union {
>> +int (*m64_map)[PCI_SRIOV_NUM_BARS]; /*Only used in powernv
>> */
>> +int last_allow_rc; /* Only used in pSeries */
>> +};
>> #endif /* CONFIG_PCI_IOV */
>> int mps;/* Maximum Payload Size */
>> struct list_head child_list;
> I don't see the point of using a union to save 4 bytes.
>
> And if you look at the current layout of the struct there's actually a 4
> byte hole after mps, so it doesn't actually save any space at all.
>
> I can remove it before applying, unless there's some compelling reason
> for it I'm not seeing.
>
> cheers
No specific reason for the union, you can go ahead and remove it before
applying.
Thanks!
Bryant
Re: [PATCH v4 0/7] SR-IOV Enablement on PowerVM
Mike Ellerman/Ben, Do you know if we can make 4.16 with this? -Bryant On 1/5/18 10:45 AM, Bryant G. Ly wrote: > This patch series will enable SR-IOV on PowerVM. A specific set of > lids for PFW/PHYP is required. They are planned to release with > 920 at the moment. > > For IBM internal testers let me know of a system you want to test on > and we can put on the lids required or we can provide a system to run > the tests. > > This patch depends on the three patches: > 988fc3ba5653278a8c14d6ccf687371775930d2b > dae7253f9f78a731755ca20c66b2d2c40b86baea > 608c0d8804ef3ca4cda8ec6ad914e47deb283d7b > > v1 - Initial Patch > v2 - Addressed Alexey and Russell's comments > v3 - Unify the call of .error_detected() > v4 - Fixed subject and change log per Bjorn's comments and > fixed Alexey's comments > > Bryant G. Ly (7): > platform/pseries: Update VF config space after EEH > linux/pci: Add uevents in AER and EEH error/resume > platforms/pseries: Set eeh_pe of EEH_PE_VF type > powerpc/kernel Add EEH operations to notify resume > powerpc/kernel: Add EEH notify resume sysfs > pseries/pci: Associate PEs to VFs in configure SR-IOV > pseries/setup: Add Initialization of VF Bars > > arch/powerpc/include/asm/eeh.h | 2 + > arch/powerpc/include/asm/pci-bridge.h| 5 +- > arch/powerpc/include/asm/pci.h | 2 + > arch/powerpc/kernel/eeh.c| 59 ++ > arch/powerpc/kernel/eeh_driver.c | 6 + > arch/powerpc/kernel/eeh_sysfs.c | 45 > arch/powerpc/kernel/pci_of_scan.c| 2 +- > arch/powerpc/platforms/powernv/eeh-powernv.c | 68 +-- > arch/powerpc/platforms/pseries/eeh_pseries.c | 137 +- > arch/powerpc/platforms/pseries/pci.c | 150 +++- > arch/powerpc/platforms/pseries/setup.c | 164 > +++ > drivers/pci/pcie/aer/aerdrv_core.c | 3 + > include/linux/pci.h | 36 ++ > 13 files changed, 611 insertions(+), 68 deletions(-) >
[PATCH v4 1/7] platform/pseries: Update VF config space after EEH
Add EEH platform operations for pseries to update VF
config space. With this change after EEH, the VF
will have updated config space for pseries platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh.c| 59 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 65 ++--
arch/powerpc/platforms/pseries/eeh_pseries.c | 26 ++-
4 files changed, 88 insertions(+), 63 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 5161c37dd039..82829c65f31a 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -297,6 +297,7 @@ int eeh_pe_reset(struct eeh_pe *pe, int option);
int eeh_pe_configure(struct eeh_pe *pe);
int eeh_pe_inject_err(struct eeh_pe *pe, int type, int func,
unsigned long addr, unsigned long mask);
+int eeh_restore_vf_config(struct pci_dn *pdn);
/**
* EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index cbca0a667682..cc649809885e 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -740,6 +740,65 @@ static void *eeh_restore_dev_state(void *data, void
*userdata)
return NULL;
}
+int eeh_restore_vf_config(struct pci_dn *pdn)
+{
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+ u32 devctl, cmd, cap2, aer_capctl;
+ int old_mps;
+
+ if (edev->pcie_cap) {
+ /* Restore MPS */
+ old_mps = (ffs(pdn->mps) - 8) << 5;
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
+ devctl |= old_mps;
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+
+ /* Disable Completion Timeout */
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCAP2,
+4, );
+ if (cap2 & 0x10) {
+ eeh_ops->read_config(pdn,
+edev->pcie_cap + PCI_EXP_DEVCTL2,
+4, );
+ cap2 |= 0x10;
+ eeh_ops->write_config(pdn,
+ edev->pcie_cap + PCI_EXP_DEVCTL2,
+ 4, cap2);
+ }
+ }
+
+ /* Enable SERR and parity checking */
+ eeh_ops->read_config(pdn, PCI_COMMAND, 2, );
+ cmd |= (PCI_COMMAND_PARITY | PCI_COMMAND_SERR);
+ eeh_ops->write_config(pdn, PCI_COMMAND, 2, cmd);
+
+ /* Enable report various errors */
+ if (edev->pcie_cap) {
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_CERE;
+ devctl |= (PCI_EXP_DEVCTL_NFERE |
+ PCI_EXP_DEVCTL_FERE |
+ PCI_EXP_DEVCTL_URRE);
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+ }
+
+ /* Enable ECRC generation and check */
+ if (edev->pcie_cap && edev->aer_cap) {
+ eeh_ops->read_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+4, _capctl);
+ aer_capctl |= (PCI_ERR_CAP_ECRC_GENE | PCI_ERR_CAP_ECRC_CHKE);
+ eeh_ops->write_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+ 4, aer_capctl);
+ }
+
+ return 0;
+}
+
/**
* pcibios_set_pcie_reset_state - Set PCI-E reset state
* @dev: pci device struct
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 961e64115d92..0665b6d03cb3 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1655,70 +1655,11 @@ static int pnv_eeh_next_error(struct eeh_pe **pe)
return ret;
}
-static int pnv_eeh_restore_vf_config(struct pci_dn *pdn)
-{
- struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
- u32 devctl, cmd, cap2, aer_capctl;
- int old_mps;
-
- if (edev->pcie_cap) {
- /* Restore MPS */
- old_mps = (ffs(pdn->mps) - 8) << 5;
- eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
-2, );
- devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
- devctl |= old_mps;
- eeh_ops->write_conf
[PATCH v4 3/7] platforms/pseries: Set eeh_pe of EEH_PE_VF type
To correctly use EEH code one has to make
sure that the EEH_PE_VF is set for dynamic created
VFs. Therefore this patch allocates an eeh_pe of
eeh type EEH_PE_VF and associates PE with parent.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci-bridge.h| 5 -
arch/powerpc/platforms/pseries/eeh_pseries.c | 17 +
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 9f66ddebb799..16d70740a76f 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -211,7 +211,10 @@ struct pci_dn {
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
boolm64_single_mode;/* Use M64 BAR in Single Mode */
#define IODA_INVALID_M64(-1)
- int (*m64_map)[PCI_SRIOV_NUM_BARS];
+ union {
+ int (*m64_map)[PCI_SRIOV_NUM_BARS]; /*Only used in powernv
*/
+ int last_allow_rc; /* Only used in pSeries */
+ };
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
struct list_head child_list;
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index a671ef4f57f5..ca6bbfd83701 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -55,9 +55,12 @@ static int ibm_get_config_addr_info;
static int ibm_get_config_addr_info2;
static int ibm_configure_pe;
+#ifdef CONFIG_PCI_IOV
void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
{
struct pci_dn *pdn = pci_get_pdn(pdev);
+ struct pci_dn *physfn_pdn;
+ struct eeh_dev *edev;
if (!pdev->is_virtfn)
return;
@@ -65,6 +68,14 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
pdn->device_id = pdev->device;
pdn->vendor_id = pdev->vendor;
pdn->class_code = pdev->class;
+ /* Last allow unfreeze return code used for retrieval
+* by user space in eeh-sysfs to show the last command
+* completion from platform
+*/
+ pdn->last_allow_rc = 0;
+ physfn_pdn = pci_get_pdn(pdev->physfn);
+ pdn->pe_number = physfn_pdn->pe_num_map[pdn->vf_index];
+ edev = pdn_to_eeh_dev(pdn);
/*
* The following operations will fail if VF's sysfs files
@@ -72,9 +83,13 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
*/
eeh_add_device_early(pdn);
eeh_add_device_late(pdev);
+ edev->pe_config_addr = (pdn->busno << 16) | (pdn->devfn << 8);
+ eeh_rmv_from_parent_pe(edev); /* Remove as it is adding to bus pe */
+ eeh_add_to_parent_pe(edev); /* Add as VF PE type */
eeh_sysfs_add_device(pdev);
}
+#endif
/*
* Buffer for reporting slot-error-detail rtas calls. Its here
@@ -141,8 +156,10 @@ static int pseries_eeh_init(void)
/* Set EEH probe mode */
eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
+#ifdef CONFIG_PCI_IOV
/* Set EEH machine dependent code */
ppc_md.pcibios_bus_add_device = pseries_pcibios_bus_add_device;
+#endif
return 0;
}
--
2.14.3 (Apple Git-98)
[PATCH v4 6/7] pseries/pci: Associate PEs to VFs in configure SR-IOV
After initial validation of SR-IOV resources, firmware will
associate PEs to the dynamic VFs created within this call. This
patch adds the association of PEs to the PF array of PE numbers
indexed by VF.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/pci.c | 150 ++-
1 file changed, 148 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 48d3af026f90..eab96637d6cf 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -59,16 +59,162 @@ DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID,
pcibios_name_device);
#endif
#ifdef CONFIG_PCI_IOV
+#define MAX_VFS_FOR_MAP_PE 256
+struct pe_map_bar_entry {
+ __be64 bar; /* Input: Virtual Function BAR */
+ __be16 rid; /* Input: Virtual Function Router ID */
+ __be16 pe_num;/* Output: Virtual Function PE Number */
+ __be32 reserved; /* Reserved Space */
+};
+
+int pseries_send_map_pe(struct pci_dev *pdev,
+ u16 num_vfs,
+ struct pe_map_bar_entry *vf_pe_array)
+{
+ struct pci_dn *pdn;
+ int rc;
+ unsigned long buid, addr;
+ int ibm_map_pes = rtas_token("ibm,open-sriov-map-pe-number");
+
+ if (ibm_map_pes == RTAS_UNKNOWN_SERVICE)
+ return -EINVAL;
+
+ pdn = pci_get_pdn(pdev);
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array,
+ RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_map_pes, 5, 1, NULL, addr,
+ BUID_HI(buid), BUID_LO(buid),
+ rtas_data_buf,
+ num_vfs * sizeof(struct pe_map_bar_entry));
+ memcpy(vf_pe_array, rtas_data_buf, RTAS_DATA_BUF_SIZE);
+ spin_unlock(_data_buf_lock);
+
+ if (rc)
+ dev_err(>dev,
+ "%s: Failed to associate pes PE#%lx, rc=%x\n",
+ __func__, addr, rc);
+
+ return rc;
+}
+
+void pseries_set_pe_num(struct pci_dev *pdev, u16 vf_index, __be16 pe_num)
+{
+ struct pci_dn *pdn;
+
+ pdn = pci_get_pdn(pdev);
+ pdn->pe_num_map[vf_index] = be16_to_cpu(pe_num);
+ dev_dbg(>dev, "VF %04x:%02x:%02x.%x associated with PE#%x\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number,
+ PCI_SLOT(pci_iov_virtfn_devfn(pdev, vf_index)),
+ PCI_FUNC(pci_iov_virtfn_devfn(pdev, vf_index)),
+ pdn->pe_num_map[vf_index]);
+}
+
+int pseries_associate_pes(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ int i, rc, vf_index;
+ struct pe_map_bar_entry *vf_pe_array;
+ struct resource *res;
+ u64 size;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ pdn = pci_get_pdn(pdev);
+ /* create firmware structure to associate pes */
+ for (vf_index = 0; vf_index < num_vfs; vf_index++) {
+ pdn->pe_num_map[vf_index] = IODA_INVALID_PE;
+ for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
+ res = >resource[i + PCI_IOV_RESOURCES];
+ if (!res->parent)
+ continue;
+ size = pcibios_iov_resource_alignment(pdev, i +
+ PCI_IOV_RESOURCES);
+ vf_pe_array[vf_index].bar =
+ cpu_to_be64(res->start + size * vf_index);
+ vf_pe_array[vf_index].rid =
+ cpu_to_be16((pci_iov_virtfn_bus(pdev, vf_index)
+ << 8) | pci_iov_virtfn_devfn(pdev,
+ vf_index));
+ vf_pe_array[vf_index].pe_num =
+ cpu_to_be16(IODA_INVALID_PE);
+ }
+ }
+
+ rc = pseries_send_map_pe(pdev, num_vfs, vf_pe_array);
+ /* Only zero is success */
+ if (!rc)
+ for (vf_index = 0; vf_index < num_vfs; vf_index++)
+ pseries_set_pe_num(pdev, vf_index,
+ vf_pe_array[vf_index].pe_num);
+
+ kfree(vf_pe_array);
+ return rc;
+}
+
+int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ intrc;
+ const int *max_vfs;
+ int max_config_vfs;
+ struct device_node *dn = pci_device_to_OF_node(pdev);
+
+ max_vfs = of_get_property(dn, "ibm,number-of-c
[PATCH v4 4/7] powerpc/kernel Add EEH operations to notify resume
When pseries SR-IOV is enabled and after a PF driver
has resumed from EEH, platform has to be notified
of the event so the child VFs can be allowed to
resume their normal recovery path.
This patch makes the EEH operation allow unfreeze
platform dependent code and adds the call to
pseries EEH code.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +-
arch/powerpc/platforms/pseries/eeh_pseries.c | 96 +++-
3 files changed, 98 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 82829c65f31a..fd37cc101f4f 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -214,6 +214,7 @@ struct eeh_ops {
int (*write_config)(struct pci_dn *pdn, int where, int size, u32 val);
int (*next_error)(struct eeh_pe **pe);
int (*restore_config)(struct pci_dn *pdn);
+ int (*notify_resume)(struct pci_dn *pdn);
};
extern int eeh_subsystem_flags;
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 0665b6d03cb3..33c86c1a1720 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1704,7 +1704,8 @@ static struct eeh_ops pnv_eeh_ops = {
.read_config= pnv_eeh_read_config,
.write_config = pnv_eeh_write_config,
.next_error = pnv_eeh_next_error,
- .restore_config = pnv_eeh_restore_config
+ .restore_config = pnv_eeh_restore_config,
+ .notify_resume = NULL
};
#ifdef CONFIG_PCI_IOV
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index ca6bbfd83701..898bb055cb19 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -749,6 +749,97 @@ static int pseries_eeh_restore_config(struct pci_dn *pdn)
return ret;
}
+#ifdef CONFIG_PCI_IOV
+int pseries_send_allow_unfreeze(struct pci_dn *pdn,
+ u16 *vf_pe_array, int cur_vfs)
+{
+ int rc;
+ int ibm_allow_unfreeze = rtas_token("ibm,open-sriov-allow-unfreeze");
+ unsigned long buid, addr;
+
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array, RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_allow_unfreeze, 5, 1, NULL,
+ addr,
+ BUID_HI(buid),
+ BUID_LO(buid),
+ rtas_data_buf, cur_vfs * sizeof(u16));
+ spin_unlock(_data_buf_lock);
+ if (rc)
+ pr_warn("%s: Failed to allow unfreeze for PHB#%x-PE#%lx,
rc=%x\n",
+ __func__,
+ pdn->phb->global_number, addr, rc);
+ return rc;
+}
+
+static int pseries_call_allow_unfreeze(struct eeh_dev *edev)
+{
+ struct pci_dn *pdn, *tmp, *parent, *physfn_pdn;
+ int cur_vfs = 0, rc = 0, vf_index, bus, devfn;
+ u16 *vf_pe_array;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+ if (pci_num_vf(edev->physfn ? edev->physfn : edev->pdev)) {
+ if (edev->pdev->is_physfn) {
+ cur_vfs = pci_num_vf(edev->pdev);
+ pdn = eeh_dev_to_pdn(edev);
+ parent = pdn->parent;
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++)
+ vf_pe_array[vf_index] =
+ cpu_to_be16(pdn->pe_num_map[vf_index]);
+ rc = pseries_send_allow_unfreeze(pdn, vf_pe_array,
+cur_vfs);
+ pdn->last_allow_rc = rc;
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++) {
+ list_for_each_entry_safe(pdn, tmp,
+>child_list,
+list) {
+ bus = pci_iov_virtfn_bus(edev->pdev,
+vf_index);
+ devfn = pci_iov_virtfn_devfn(edev->pdev,
+vf_index);
+ if (pdn->busno != bus ||
+ pdn->devfn != devfn)
+
[PATCH v4 2/7] linux/pci: Add uevents in AER and EEH error/resume
Devices can go offline when erors reported. This
patch adds a change to the kernel object and lets udev
know of error. When device resumes, a change is also set
reporting device as online. Therefore, EEH and AER events
are better propagated to user space for PCI devices in
all arches.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelg...@google.com>
---
arch/powerpc/kernel/eeh_driver.c | 6 ++
drivers/pci/pcie/aer/aerdrv_core.c | 3 +++
include/linux/pci.h| 36
3 files changed, 45 insertions(+)
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3c0fa99c5533..beea2182d754 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -228,6 +228,7 @@ static void *eeh_report_error(void *data, void *userdata)
edev->in_error = true;
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_NONE);
return NULL;
}
@@ -381,6 +382,10 @@ static void *eeh_report_resume(void *data, void *userdata)
driver->err_handler->resume(dev);
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
+#ifdef CONFIG_PCI_IOV
+ eeh_ops->notify_resume(eeh_dev_to_pdn(edev));
+#endif
return NULL;
}
@@ -416,6 +421,7 @@ static void *eeh_report_failure(void *data, void *userdata)
driver->err_handler->error_detected(dev, pci_channel_io_perm_failure);
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_DISCONNECT);
return NULL;
}
diff --git a/drivers/pci/pcie/aer/aerdrv_core.c
b/drivers/pci/pcie/aer/aerdrv_core.c
index 744805232155..8d7448063fd1 100644
--- a/drivers/pci/pcie/aer/aerdrv_core.c
+++ b/drivers/pci/pcie/aer/aerdrv_core.c
@@ -278,6 +278,7 @@ static int report_error_detected(struct pci_dev *dev, void
*data)
} else {
err_handler = dev->driver->err_handler;
vote = err_handler->error_detected(dev, result_data->state);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_NONE);
}
result_data->result = merge_result(result_data->result, vote);
@@ -341,6 +342,7 @@ static int report_resume(struct pci_dev *dev, void *data)
err_handler = dev->driver->err_handler;
err_handler->resume(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
out:
device_unlock(>dev);
return 0;
@@ -541,6 +543,7 @@ static void do_recovery(struct pci_dev *dev, int severity)
return;
failed:
+ pci_uevent_ers(dev, PCI_ERS_RESULT_DISCONNECT);
/* TODO: Should kernel panic here? */
dev_info(>dev, "AER: Device recovery failed\n");
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index e3e94467687a..405630441b74 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2277,6 +2277,42 @@ static inline bool pci_is_thunderbolt_attached(struct
pci_dev *pdev)
return false;
}
+/**
+ * pci_uevent_ers - emit a uevent during recovery path of pci device
+ * @pdev: pci device to check
+ * @err_type: type of error event
+ *
+ */
+static inline void pci_uevent_ers(struct pci_dev *pdev,
+ enum pci_ers_result err_type)
+{
+ int idx = 0;
+ char *envp[3];
+
+ switch (err_type) {
+ case PCI_ERS_RESULT_NONE:
+ case PCI_ERS_RESULT_CAN_RECOVER:
+ envp[idx++] = "ERROR_EVENT=BEGIN_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=0";
+ break;
+ case PCI_ERS_RESULT_RECOVERED:
+ envp[idx++] = "ERROR_EVENT=SUCCESSFUL_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=1";
+ break;
+ case PCI_ERS_RESULT_DISCONNECT:
+ envp[idx++] = "ERROR_EVENT=FAILED_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=0";
+ break;
+ default:
+ break;
+ }
+
+ if (idx > 0) {
+ envp[idx++] = NULL;
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
+ }
+}
+
/* provide the legacy pci_dma_* API */
#include
--
2.14.3 (Apple Git-98)
[PATCH v4 0/7] SR-IOV Enablement on PowerVM
This patch series will enable SR-IOV on PowerVM. A specific set of lids for PFW/PHYP is required. They are planned to release with 920 at the moment. For IBM internal testers let me know of a system you want to test on and we can put on the lids required or we can provide a system to run the tests. This patch depends on the three patches: 988fc3ba5653278a8c14d6ccf687371775930d2b dae7253f9f78a731755ca20c66b2d2c40b86baea 608c0d8804ef3ca4cda8ec6ad914e47deb283d7b v1 - Initial Patch v2 - Addressed Alexey and Russell's comments v3 - Unify the call of .error_detected() v4 - Fixed subject and change log per Bjorn's comments and fixed Alexey's comments Bryant G. Ly (7): platform/pseries: Update VF config space after EEH linux/pci: Add uevents in AER and EEH error/resume platforms/pseries: Set eeh_pe of EEH_PE_VF type powerpc/kernel Add EEH operations to notify resume powerpc/kernel: Add EEH notify resume sysfs pseries/pci: Associate PEs to VFs in configure SR-IOV pseries/setup: Add Initialization of VF Bars arch/powerpc/include/asm/eeh.h | 2 + arch/powerpc/include/asm/pci-bridge.h| 5 +- arch/powerpc/include/asm/pci.h | 2 + arch/powerpc/kernel/eeh.c| 59 ++ arch/powerpc/kernel/eeh_driver.c | 6 + arch/powerpc/kernel/eeh_sysfs.c | 45 arch/powerpc/kernel/pci_of_scan.c| 2 +- arch/powerpc/platforms/powernv/eeh-powernv.c | 68 +-- arch/powerpc/platforms/pseries/eeh_pseries.c | 137 +- arch/powerpc/platforms/pseries/pci.c | 150 +++- arch/powerpc/platforms/pseries/setup.c | 164 +++ drivers/pci/pcie/aer/aerdrv_core.c | 3 + include/linux/pci.h | 36 ++ 13 files changed, 611 insertions(+), 68 deletions(-) -- 2.14.3 (Apple Git-98)
[PATCH v4 7/7] pseries/setup: Add Initialization of VF Bars
When enabling SR-IOV in pseries platform,
the VF bar properties for a PF are reported on
the device node in the device tree.
This patch adds the IOV Bar resources to Linux
structures from the device tree for later use
when configuring SR-IOV by PF driver.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci.h | 2 +
arch/powerpc/kernel/pci_of_scan.c | 2 +-
arch/powerpc/platforms/pseries/setup.c | 164 +
3 files changed, 167 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 8dc32eacc97c..d82802ff5088 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -121,6 +121,8 @@ extern int remove_phb_dynamic(struct pci_controller *phb);
extern struct pci_dev *of_create_pci_dev(struct device_node *node,
struct pci_bus *bus, int devfn);
+extern unsigned int pci_parse_of_flags(u32 addr0, int bridge);
+
extern void of_scan_pci_bridge(struct pci_dev *dev);
extern void of_scan_bus(struct device_node *node, struct pci_bus *bus);
diff --git a/arch/powerpc/kernel/pci_of_scan.c
b/arch/powerpc/kernel/pci_of_scan.c
index 0d790f8432d2..20ceec4a5f5e 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -38,7 +38,7 @@ static u32 get_int_prop(struct device_node *np, const char
*name, u32 def)
* @addr0: value of 1st cell of a device tree PCI address.
* @bridge: Set this flag if the address is from a bridge 'ranges' property
*/
-static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
+unsigned int pci_parse_of_flags(u32 addr0, int bridge)
{
unsigned int flags = 0;
diff --git a/arch/powerpc/platforms/pseries/setup.c
b/arch/powerpc/platforms/pseries/setup.c
index 1d6e2de2445c..e8f523cb5526 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -459,6 +459,162 @@ static void __init find_and_init_phbs(void)
of_pci_check_probe_only();
}
+#ifdef CONFIG_PCI_IOV
+enum rtas_iov_fw_value_map {
+ NUM_RES_PROPERTY = 0, /* Number of Resources */
+ LOW_INT = 1, /* Lowest 32 bits of Address */
+ START_OF_ENTRIES = 2, /* Always start of entry */
+ APERTURE_PROPERTY = 2, /* Start of entry+ to Aperture Size */
+ WDW_SIZE_PROPERTY = 4, /* Start of entry+ to Window Size */
+ NEXT_ENTRY= 7 /* Go to next entry on array */
+};
+
+enum get_iov_fw_value_index {
+ BAR_ADDRS = 1,/* Get Bar Address */
+ APERTURE_SIZE = 2,/* Get Aperture Size */
+ WDW_SIZE = 3 /* Get Window Size */
+};
+
+resource_size_t pseries_get_iov_fw_value(struct pci_dev *dev, int resno,
+enum get_iov_fw_value_index value)
+{
+ const int *indexes;
+ struct device_node *dn = pci_device_to_OF_node(dev);
+ int i, num_res, ret = 0;
+
+ indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
+ if (!indexes)
+ return 0;
+
+ /*
+* First element in the array is the number of Bars
+* returned. Search through the list to find the matching
+* bar
+*/
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ if (resno >= num_res)
+ return 0; /* or an errror */
+
+ i = START_OF_ENTRIES + NEXT_ENTRY * resno;
+ switch (value) {
+ case BAR_ADDRS:
+ ret = of_read_number([i], 2);
+ break;
+ case APERTURE_SIZE:
+ ret = of_read_number([i + APERTURE_PROPERTY], 2);
+ break;
+ case WDW_SIZE:
+ ret = of_read_number([i + WDW_SIZE_PROPERTY], 2);
+ break;
+ }
+
+ return ret;
+}
+
+void of_pci_set_vf_bar_size(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *res;
+ resource_size_t base, size;
+ int i, r, num_res;
+
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ num_res = min_t(int, num_res, PCI_SRIOV_NUM_BARS);
+ for (i = START_OF_ENTRIES, r = 0; r < num_res && r < PCI_SRIOV_NUM_BARS;
+i += NEXT_ENTRY, r++) {
+ res = >resource[r + PCI_IOV_RESOURCES];
+ base = of_read_number([i], 2);
+ size = of_read_number([i + APERTURE_PROPERTY], 2);
+ res->flags = pci_parse_of_flags(of_read_number
+ ([i + LOW_INT], 1), 0);
+ res->flags |= (IORESOURCE_MEM_64 | IORESOURCE_PCI_FIXED);
+ res->name = pci_name(dev);
+ res->start = base;
+ res->end = base + size - 1;
+ }
+}
+
+void of_pci_parse_iov_addrs(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *
[PATCH v4 5/7] powerpc/kernel: Add EEH notify resume sysfs
Introduce a method for notify resume to be
called from sysfs. In this patch one can
now call notify resume from sysfs when
is supported by platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_sysfs.c | 45 +
1 file changed, 45 insertions(+)
diff --git a/arch/powerpc/kernel/eeh_sysfs.c b/arch/powerpc/kernel/eeh_sysfs.c
index 797549289798..9c513abc102c 100644
--- a/arch/powerpc/kernel/eeh_sysfs.c
+++ b/arch/powerpc/kernel/eeh_sysfs.c
@@ -90,6 +90,38 @@ static ssize_t eeh_pe_state_store(struct device *dev,
static DEVICE_ATTR_RW(eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+static ssize_t eeh_notify_resume_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ pdn = pci_get_pdn(pdev);
+ return sprintf(buf, "%d\n", pdn->last_allow_rc);
+}
+
+static ssize_t eeh_notify_resume_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ if (eeh_ops->notify_resume(pci_get_pdn(pdev)))
+ return -EIO;
+ return count;
+}
+static DEVICE_ATTR_RW(eeh_notify_resume);
+#endif
+
void eeh_sysfs_add_device(struct pci_dev *pdev)
{
struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
@@ -105,6 +137,13 @@ void eeh_sysfs_add_device(struct pci_dev *pdev)
rc += device_create_file(>dev, _attr_eeh_pe_config_addr);
rc += device_create_file(>dev, _attr_eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ rc += device_create_file(>dev,
+_attr_eeh_notify_resume);
+#endif
if (rc)
pr_warn("EEH: Unable to create sysfs entries\n");
else if (edev)
@@ -128,6 +167,12 @@ void eeh_sysfs_remove_device(struct pci_dev *pdev)
device_remove_file(>dev, _attr_eeh_mode);
device_remove_file(>dev, _attr_eeh_pe_config_addr);
device_remove_file(>dev, _attr_eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ device_remove_file(>dev, _attr_eeh_notify_resume);
+#endif
if (edev)
edev->mode &= ~EEH_DEV_SYSFS;
--
2.14.3 (Apple Git-98)
[PATCH v3 5/7] powerpc/kernel: Add EEH notify resume sysfs
Introduce a method for notify resume to be
called from sysfs. In this patch one can
now call notify resume from sysfs when
is supported by platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_sysfs.c | 46 -
1 file changed, 45 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/eeh_sysfs.c b/arch/powerpc/kernel/eeh_sysfs.c
index 797549289798..344e7607f2e0 100644
--- a/arch/powerpc/kernel/eeh_sysfs.c
+++ b/arch/powerpc/kernel/eeh_sysfs.c
@@ -90,6 +90,38 @@ static ssize_t eeh_pe_state_store(struct device *dev,
static DEVICE_ATTR_RW(eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+static ssize_t eeh_notify_resume_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ pdn = pci_get_pdn(pdev);
+ return sprintf(buf, "%d\n", pdn->last_allow_rc);
+}
+
+static ssize_t eeh_notify_resume_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ if (eeh_ops->notify_resume(pci_get_pdn(pdev)))
+ return -EIO;
+ return count;
+}
+static DEVICE_ATTR_RW(eeh_notify_resume);
+#endif
+
void eeh_sysfs_add_device(struct pci_dev *pdev)
{
struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
@@ -104,7 +136,13 @@ void eeh_sysfs_add_device(struct pci_dev *pdev)
rc += device_create_file(>dev, _attr_eeh_mode);
rc += device_create_file(>dev, _attr_eeh_pe_config_addr);
rc += device_create_file(>dev, _attr_eeh_pe_state);
-
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ rc += device_create_file(>dev,
+_attr_eeh_notify_resume);
+#endif
if (rc)
pr_warn("EEH: Unable to create sysfs entries\n");
else if (edev)
@@ -128,6 +166,12 @@ void eeh_sysfs_remove_device(struct pci_dev *pdev)
device_remove_file(>dev, _attr_eeh_mode);
device_remove_file(>dev, _attr_eeh_pe_config_addr);
device_remove_file(>dev, _attr_eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ device_remove_file(>dev, _attr_eeh_notify_resume);
+#endif
if (edev)
edev->mode &= ~EEH_DEV_SYSFS;
--
2.14.3 (Apple Git-98)
[PATCH v3 6/7] pseries/pci: Associate PEs to VFs in configure SR-IOV
After initial validation of SR-IOV resources, firmware will
associate PEs to the dynamic VFs created within this call. This
patch adds the association of PEs to the PF array of PE numbers
indexed by VF.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/pci.c | 151 ++-
1 file changed, 148 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 48d3af026f90..00e6475004ff 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -57,18 +57,163 @@ void pcibios_name_device(struct pci_dev *dev)
}
DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
#endif
-
#ifdef CONFIG_PCI_IOV
+#define MAX_VFS_FOR_MAP_PE 256
+struct pe_map_bar_entry {
+ __be64 bar; /* Input: Virtual Function BAR */
+ __be16 rid; /* Input: Virtual Function Router ID */
+ __be16 pe_num;/* Output: Virtual Function PE Number */
+ __be32 reserved; /* Reserved Space */
+};
+
+int pseries_send_map_pe(struct pci_dev *pdev,
+ u16 num_vfs,
+ struct pe_map_bar_entry *vf_pe_array)
+{
+ struct pci_dn *pdn;
+ int rc;
+ unsigned long buid, addr;
+ int ibm_map_pes = rtas_token("ibm,open-sriov-map-pe-number");
+
+ if (ibm_map_pes == RTAS_UNKNOWN_SERVICE)
+ return -EINVAL;
+
+ pdn = pci_get_pdn(pdev);
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array,
+ RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_map_pes, 5, 1, NULL, addr,
+ BUID_HI(buid), BUID_LO(buid),
+ rtas_data_buf,
+ num_vfs * sizeof(struct pe_map_bar_entry));
+ memcpy(vf_pe_array, rtas_data_buf, RTAS_DATA_BUF_SIZE);
+ spin_unlock(_data_buf_lock);
+
+ if (rc)
+ dev_err(>dev,
+ "%s: Failed to associate pes PE#%lx, rc=%x\n",
+ __func__, addr, rc);
+
+ return rc;
+}
+
+void pseries_set_pe_num(struct pci_dev *pdev, u16 vf_index, __be16 pe_num)
+{
+ struct pci_dn *pdn;
+
+ pdn = pci_get_pdn(pdev);
+ pdn->pe_num_map[vf_index] = be16_to_cpu(pe_num);
+ dev_dbg(>dev, "VF %04x:%02x:%02x.%x associated with PE#%x\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number,
+ PCI_SLOT(pci_iov_virtfn_devfn(pdev, vf_index)),
+ PCI_FUNC(pci_iov_virtfn_devfn(pdev, vf_index)),
+ pdn->pe_num_map[vf_index]);
+}
+
+int pseries_associate_pes(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ int i, rc, vf_index;
+ struct pe_map_bar_entry *vf_pe_array;
+ struct resource *res;
+ u64 size;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ pdn = pci_get_pdn(pdev);
+ /* create firmware structure to associate pes */
+ for (vf_index = 0; vf_index < num_vfs; vf_index++) {
+ pdn->pe_num_map[vf_index] = IODA_INVALID_PE;
+ for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
+ res = >resource[i + PCI_IOV_RESOURCES];
+ if (!res->parent)
+ continue;
+ size = pcibios_iov_resource_alignment(pdev, i +
+ PCI_IOV_RESOURCES);
+ vf_pe_array[vf_index].bar =
+ cpu_to_be64(res->start + size * vf_index);
+ vf_pe_array[vf_index].rid =
+ cpu_to_be16((pci_iov_virtfn_bus(pdev, vf_index)
+ << 8) | pci_iov_virtfn_devfn(pdev,
+ vf_index));
+ vf_pe_array[vf_index].pe_num =
+ cpu_to_be16(IODA_INVALID_PE);
+ }
+ }
+
+ rc = pseries_send_map_pe(pdev, num_vfs, vf_pe_array);
+ /* Only zero is success */
+ if (!rc)
+ for (vf_index = 0; vf_index < num_vfs; vf_index++)
+ pseries_set_pe_num(pdev, vf_index,
+ vf_pe_array[vf_index].pe_num);
+
+ kfree(vf_pe_array);
+ return rc;
+}
+
+int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ intrc;
+ const int *max_vfs;
+ int max_config_vfs;
+ struct device_node *dn = pci_device_to_OF_node(pdev);
+
[PATCH v3 1/7] platform/pseries: Update VF config space after EEH
Add EEH platform operations for pseries to update VF
config space. With this change after EEH, the VF
will have updated config space for pseries platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh.c| 59 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 65 ++--
arch/powerpc/platforms/pseries/eeh_pseries.c | 26 ++-
4 files changed, 88 insertions(+), 63 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 5161c37dd039..82829c65f31a 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -297,6 +297,7 @@ int eeh_pe_reset(struct eeh_pe *pe, int option);
int eeh_pe_configure(struct eeh_pe *pe);
int eeh_pe_inject_err(struct eeh_pe *pe, int type, int func,
unsigned long addr, unsigned long mask);
+int eeh_restore_vf_config(struct pci_dn *pdn);
/**
* EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index cbca0a667682..cc649809885e 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -740,6 +740,65 @@ static void *eeh_restore_dev_state(void *data, void
*userdata)
return NULL;
}
+int eeh_restore_vf_config(struct pci_dn *pdn)
+{
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+ u32 devctl, cmd, cap2, aer_capctl;
+ int old_mps;
+
+ if (edev->pcie_cap) {
+ /* Restore MPS */
+ old_mps = (ffs(pdn->mps) - 8) << 5;
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
+ devctl |= old_mps;
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+
+ /* Disable Completion Timeout */
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCAP2,
+4, );
+ if (cap2 & 0x10) {
+ eeh_ops->read_config(pdn,
+edev->pcie_cap + PCI_EXP_DEVCTL2,
+4, );
+ cap2 |= 0x10;
+ eeh_ops->write_config(pdn,
+ edev->pcie_cap + PCI_EXP_DEVCTL2,
+ 4, cap2);
+ }
+ }
+
+ /* Enable SERR and parity checking */
+ eeh_ops->read_config(pdn, PCI_COMMAND, 2, );
+ cmd |= (PCI_COMMAND_PARITY | PCI_COMMAND_SERR);
+ eeh_ops->write_config(pdn, PCI_COMMAND, 2, cmd);
+
+ /* Enable report various errors */
+ if (edev->pcie_cap) {
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_CERE;
+ devctl |= (PCI_EXP_DEVCTL_NFERE |
+ PCI_EXP_DEVCTL_FERE |
+ PCI_EXP_DEVCTL_URRE);
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+ }
+
+ /* Enable ECRC generation and check */
+ if (edev->pcie_cap && edev->aer_cap) {
+ eeh_ops->read_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+4, _capctl);
+ aer_capctl |= (PCI_ERR_CAP_ECRC_GENE | PCI_ERR_CAP_ECRC_CHKE);
+ eeh_ops->write_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+ 4, aer_capctl);
+ }
+
+ return 0;
+}
+
/**
* pcibios_set_pcie_reset_state - Set PCI-E reset state
* @dev: pci device struct
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 961e64115d92..0665b6d03cb3 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1655,70 +1655,11 @@ static int pnv_eeh_next_error(struct eeh_pe **pe)
return ret;
}
-static int pnv_eeh_restore_vf_config(struct pci_dn *pdn)
-{
- struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
- u32 devctl, cmd, cap2, aer_capctl;
- int old_mps;
-
- if (edev->pcie_cap) {
- /* Restore MPS */
- old_mps = (ffs(pdn->mps) - 8) << 5;
- eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
-2, );
- devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
- devctl |= old_mps;
- eeh_ops->write_conf
[PATCH v3 7/7] pseries/setup: Add Initialization of VF Bars
When enabling SR-IOV in pseries platform,
the VF bar properties for a PF are reported on
the device node in the device tree.
This patch adds the IOV Bar resources to Linux
structures from the device tree for later use
when configuring SR-IOV by PF driver.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci.h | 2 +
arch/powerpc/kernel/pci_of_scan.c | 2 +-
arch/powerpc/platforms/pseries/setup.c | 164 +
3 files changed, 167 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 8dc32eacc97c..d82802ff5088 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -121,6 +121,8 @@ extern int remove_phb_dynamic(struct pci_controller *phb);
extern struct pci_dev *of_create_pci_dev(struct device_node *node,
struct pci_bus *bus, int devfn);
+extern unsigned int pci_parse_of_flags(u32 addr0, int bridge);
+
extern void of_scan_pci_bridge(struct pci_dev *dev);
extern void of_scan_bus(struct device_node *node, struct pci_bus *bus);
diff --git a/arch/powerpc/kernel/pci_of_scan.c
b/arch/powerpc/kernel/pci_of_scan.c
index 0d790f8432d2..20ceec4a5f5e 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -38,7 +38,7 @@ static u32 get_int_prop(struct device_node *np, const char
*name, u32 def)
* @addr0: value of 1st cell of a device tree PCI address.
* @bridge: Set this flag if the address is from a bridge 'ranges' property
*/
-static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
+unsigned int pci_parse_of_flags(u32 addr0, int bridge)
{
unsigned int flags = 0;
diff --git a/arch/powerpc/platforms/pseries/setup.c
b/arch/powerpc/platforms/pseries/setup.c
index 5f1beb8367ac..1f8f70ec50d1 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -459,6 +459,162 @@ static void __init find_and_init_phbs(void)
of_pci_check_probe_only();
}
+#ifdef CONFIG_PCI_IOV
+enum rtas_iov_fw_value_map {
+ NUM_RES_PROPERTY = 0, /* Number of Resources */
+ LOW_INT = 1, /* Lowest 32 bits of Address */
+ START_OF_ENTRIES = 2, /* Always start of entry */
+ APERTURE_PROPERTY = 2, /* Start of entry+ to Aperture Size */
+ WDW_SIZE_PROPERTY = 4, /* Start of entry+ to Window Size */
+ NEXT_ENTRY= 7 /* Go to next entry on array */
+};
+
+enum get_iov_fw_value_index {
+ BAR_ADDRS = 1,/* Get Bar Address */
+ APERTURE_SIZE = 2,/* Get Aperture Size */
+ WDW_SIZE = 3 /* Get Window Size */
+};
+
+resource_size_t pseries_get_iov_fw_value(struct pci_dev *dev, int resno,
+enum get_iov_fw_value_index value)
+{
+ const int *indexes;
+ struct device_node *dn = pci_device_to_OF_node(dev);
+ int i, num_res, ret = 0;
+
+ indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
+ if (!indexes)
+ return 0;
+
+ /*
+* First element in the array is the number of Bars
+* returned. Search through the list to find the matching
+* bar
+*/
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ if (resno >= num_res)
+ return 0; /* or an errror */
+
+ i = START_OF_ENTRIES + NEXT_ENTRY * resno;
+ switch (value) {
+ case BAR_ADDRS:
+ ret = of_read_number([i], 2);
+ break;
+ case APERTURE_SIZE:
+ ret = of_read_number([i + APERTURE_PROPERTY], 2);
+ break;
+ case WDW_SIZE:
+ ret = of_read_number([i + WDW_SIZE_PROPERTY], 2);
+ break;
+ }
+
+ return ret;
+}
+
+void of_pci_set_vf_bar_size(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *res;
+ resource_size_t base, size;
+ int i, r, num_res;
+
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ num_res = min_t(int, num_res, PCI_SRIOV_NUM_BARS);
+ for (i = START_OF_ENTRIES, r = 0; r < num_res && r < PCI_SRIOV_NUM_BARS;
+i += NEXT_ENTRY, r++) {
+ res = >resource[r + PCI_IOV_RESOURCES];
+ base = of_read_number([i], 2);
+ size = of_read_number([i + APERTURE_PROPERTY], 2);
+ res->flags = pci_parse_of_flags(of_read_number
+ ([i + LOW_INT], 1), 0);
+ res->flags |= (IORESOURCE_MEM_64 | IORESOURCE_PCI_FIXED);
+ res->name = pci_name(dev);
+ res->start = base;
+ res->end = base + size - 1;
+ }
+}
+
+void of_pci_parse_iov_addrs(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *
[PATCH v3 4/7] powerpc/kernel Add EEH operations to notify resume
When pseries SR-IOV is enabled and after a PF driver
has resumed from EEH, platform has to be notified
of the event so the child VFs can be allowed to
resume their normal recovery path.
This patch makes the EEH operation allow unfreeze
platform dependent code and adds the call to
pseries EEH code.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +-
arch/powerpc/platforms/pseries/eeh_pseries.c | 94 +++-
3 files changed, 96 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 82829c65f31a..fd37cc101f4f 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -214,6 +214,7 @@ struct eeh_ops {
int (*write_config)(struct pci_dn *pdn, int where, int size, u32 val);
int (*next_error)(struct eeh_pe **pe);
int (*restore_config)(struct pci_dn *pdn);
+ int (*notify_resume)(struct pci_dn *pdn);
};
extern int eeh_subsystem_flags;
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 0665b6d03cb3..33c86c1a1720 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1704,7 +1704,8 @@ static struct eeh_ops pnv_eeh_ops = {
.read_config= pnv_eeh_read_config,
.write_config = pnv_eeh_write_config,
.next_error = pnv_eeh_next_error,
- .restore_config = pnv_eeh_restore_config
+ .restore_config = pnv_eeh_restore_config,
+ .notify_resume = NULL
};
#ifdef CONFIG_PCI_IOV
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 686e9c7feab5..bdd7be52b12d 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -744,6 +744,97 @@ static int pseries_eeh_restore_config(struct pci_dn *pdn)
return ret;
}
+#ifdef CONFIG_PCI_IOV
+int pseries_send_allow_unfreeze(struct pci_dn *pdn,
+ u16 *vf_pe_array, int cur_vfs)
+{
+ int rc;
+ int ibm_allow_unfreeze = rtas_token("ibm,open-sriov-allow-unfreeze");
+ unsigned long buid, addr;
+
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array, RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_allow_unfreeze, 5, 1, NULL,
+ addr,
+ BUID_HI(buid),
+ BUID_LO(buid),
+ rtas_data_buf, cur_vfs * sizeof(u16));
+ spin_unlock(_data_buf_lock);
+ if (rc)
+ pr_warn("%s: Failed to allow unfreeze for PHB#%x-PE#%lx,
rc=%x\n",
+ __func__,
+ pdn->phb->global_number, addr, rc);
+ return rc;
+}
+
+static int pseries_call_allow_unfreeze(struct eeh_dev *edev)
+{
+ struct pci_dn *pdn, *tmp, *parent, *physfn_pdn;
+ int cur_vfs = 0, rc = 0, vf_index, bus, devfn;
+ u16 *vf_pe_array;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+ if (pci_num_vf(edev->physfn ? edev->physfn : edev->pdev)) {
+ if (edev->pdev->is_physfn) {
+ cur_vfs = pci_num_vf(edev->pdev);
+ pdn = eeh_dev_to_pdn(edev);
+ parent = pdn->parent;
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++)
+ vf_pe_array[vf_index] =
+ cpu_to_be16(pdn->pe_num_map[vf_index]);
+ rc = pseries_send_allow_unfreeze(pdn, vf_pe_array,
+cur_vfs);
+ pdn->last_allow_rc = rc;
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++) {
+ list_for_each_entry_safe(pdn, tmp,
+>child_list,
+list) {
+ bus = pci_iov_virtfn_bus(edev->pdev,
+vf_index);
+ devfn = pci_iov_virtfn_devfn(edev->pdev,
+vf_index);
+ if (pdn->busno != bus ||
+ pdn->devfn != devfn)
+
[PATCH v3 3/7] platforms/pseries: Set eeh_pe of EEH_PE_VF type
To correctly use EEH code one has to make
sure that the EEH_PE_VF is set for dynamic created
VFs. Therefore this patch allocates an eeh_pe of
eeh type EEH_PE_VF and associates PE with parent.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci-bridge.h| 5 -
arch/powerpc/platforms/pseries/eeh_pseries.c | 14 +-
2 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 9f66ddebb799..16d70740a76f 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -211,7 +211,10 @@ struct pci_dn {
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
boolm64_single_mode;/* Use M64 BAR in Single Mode */
#define IODA_INVALID_M64(-1)
- int (*m64_map)[PCI_SRIOV_NUM_BARS];
+ union {
+ int (*m64_map)[PCI_SRIOV_NUM_BARS]; /*Only used in powernv
*/
+ int last_allow_rc; /* Only used in pSeries */
+ };
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
struct list_head child_list;
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index a671ef4f57f5..686e9c7feab5 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -58,6 +58,8 @@ static int ibm_configure_pe;
void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
{
struct pci_dn *pdn = pci_get_pdn(pdev);
+ struct pci_dn *physfn_pdn;
+ struct eeh_dev *edev;
if (!pdev->is_virtfn)
return;
@@ -65,6 +67,14 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
pdn->device_id = pdev->device;
pdn->vendor_id = pdev->vendor;
pdn->class_code = pdev->class;
+ /* Last allow unfreeze return code used for retrieval
+* by user space in eeh-sysfs to show the last command
+* completion from platform
+*/
+ pdn->last_allow_rc = 0;
+ physfn_pdn = pci_get_pdn(pdev->physfn);
+ pdn->pe_number = physfn_pdn->pe_num_map[pdn->vf_index];
+ edev = pdn_to_eeh_dev(pdn);
/*
* The following operations will fail if VF's sysfs files
@@ -72,8 +82,10 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
*/
eeh_add_device_early(pdn);
eeh_add_device_late(pdev);
+ edev->pe_config_addr = (pdn->busno << 16) | (pdn->devfn << 8);
+ eeh_rmv_from_parent_pe(edev); /* Remove as it is adding to bus pe */
+ eeh_add_to_parent_pe(edev); /* Add as VF PE type */
eeh_sysfs_add_device(pdev);
-
}
/*
--
2.14.3 (Apple Git-98)
[PATCH v3 2/7] powerpc/kernel: Add uevents in EEH error/resume
Devices can go offline when EEH is reported. This patch adds
a change to the kernel object and lets udev know of error.
When device resumes a change is also set reporting device as
online. Therefore, EEH events are better propagated to user
space for devices in powerpc arch.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_driver.c | 8 ++--
drivers/pci/pcie/aer/aerdrv_core.c | 3 +++
include/linux/pci.h| 36
3 files changed, 45 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3c0fa99c5533..c2945b91b628 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -228,6 +228,7 @@ static void *eeh_report_error(void *data, void *userdata)
edev->in_error = true;
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_NONE);
return NULL;
}
@@ -379,8 +380,11 @@ static void *eeh_report_resume(void *data, void *userdata)
}
driver->err_handler->resume(dev);
-
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
+#ifdef CONFIG_PCI_IOV
+ eeh_ops->notify_resume(eeh_dev_to_pdn(edev));
+#endif
return NULL;
}
@@ -414,8 +418,8 @@ static void *eeh_report_failure(void *data, void *userdata)
}
driver->err_handler->error_detected(dev, pci_channel_io_perm_failure);
-
eeh_pcid_put(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_DISCONNECT);
return NULL;
}
diff --git a/drivers/pci/pcie/aer/aerdrv_core.c
b/drivers/pci/pcie/aer/aerdrv_core.c
index 744805232155..8d7448063fd1 100644
--- a/drivers/pci/pcie/aer/aerdrv_core.c
+++ b/drivers/pci/pcie/aer/aerdrv_core.c
@@ -278,6 +278,7 @@ static int report_error_detected(struct pci_dev *dev, void
*data)
} else {
err_handler = dev->driver->err_handler;
vote = err_handler->error_detected(dev, result_data->state);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_NONE);
}
result_data->result = merge_result(result_data->result, vote);
@@ -341,6 +342,7 @@ static int report_resume(struct pci_dev *dev, void *data)
err_handler = dev->driver->err_handler;
err_handler->resume(dev);
+ pci_uevent_ers(dev, PCI_ERS_RESULT_RECOVERED);
out:
device_unlock(>dev);
return 0;
@@ -541,6 +543,7 @@ static void do_recovery(struct pci_dev *dev, int severity)
return;
failed:
+ pci_uevent_ers(dev, PCI_ERS_RESULT_DISCONNECT);
/* TODO: Should kernel panic here? */
dev_info(>dev, "AER: Device recovery failed\n");
}
diff --git a/include/linux/pci.h b/include/linux/pci.h
index e3e94467687a..405630441b74 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -2277,6 +2277,42 @@ static inline bool pci_is_thunderbolt_attached(struct
pci_dev *pdev)
return false;
}
+/**
+ * pci_uevent_ers - emit a uevent during recovery path of pci device
+ * @pdev: pci device to check
+ * @err_type: type of error event
+ *
+ */
+static inline void pci_uevent_ers(struct pci_dev *pdev,
+ enum pci_ers_result err_type)
+{
+ int idx = 0;
+ char *envp[3];
+
+ switch (err_type) {
+ case PCI_ERS_RESULT_NONE:
+ case PCI_ERS_RESULT_CAN_RECOVER:
+ envp[idx++] = "ERROR_EVENT=BEGIN_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=0";
+ break;
+ case PCI_ERS_RESULT_RECOVERED:
+ envp[idx++] = "ERROR_EVENT=SUCCESSFUL_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=1";
+ break;
+ case PCI_ERS_RESULT_DISCONNECT:
+ envp[idx++] = "ERROR_EVENT=FAILED_RECOVERY";
+ envp[idx++] = "DEVICE_ONLINE=0";
+ break;
+ default:
+ break;
+ }
+
+ if (idx > 0) {
+ envp[idx++] = NULL;
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
+ }
+}
+
/* provide the legacy pci_dma_* API */
#include
--
2.14.3 (Apple Git-98)
[PATCH v3 0/7] SR-IOV Enablement on PowerVM
This patch series will enable SR-IOV on PowerVM. A specific set of lids for PFW/PHYP is required. They are planned to release with 920 at the moment. For IBM internal testers let me know of a system you want to test on and we can put on the lids required or we can provide a system to run the tests. This patch depends on the three patches: 988fc3ba5653278a8c14d6ccf687371775930d2b dae7253f9f78a731755ca20c66b2d2c40b86baea 608c0d8804ef3ca4cda8ec6ad914e47deb283d7b v1 - Initial Patch v2 - Addressed Alexey and Russell's comments v3 - Unify the call of .error_detected() Bryant G. Ly (7): platform/pseries: Update VF config space after EEH powerpc/kernel: Add uevents in EEH error/resume platforms/pseries: Set eeh_pe of EEH_PE_VF type powerpc/kernel Add EEH operations to notify resume powerpc/kernel: Add EEH notify resume sysfs pseries/pci: Associate PEs to VFs in configure SR-IOV pseries/setup: Add Initialization of VF Bars arch/powerpc/include/asm/eeh.h | 2 + arch/powerpc/include/asm/pci-bridge.h| 5 +- arch/powerpc/include/asm/pci.h | 2 + arch/powerpc/kernel/eeh.c| 59 ++ arch/powerpc/kernel/eeh_driver.c | 8 +- arch/powerpc/kernel/eeh_sysfs.c | 46 +++- arch/powerpc/kernel/pci_of_scan.c| 2 +- arch/powerpc/platforms/powernv/eeh-powernv.c | 68 +-- arch/powerpc/platforms/pseries/eeh_pseries.c | 132 - arch/powerpc/platforms/pseries/pci.c | 151 +++- arch/powerpc/platforms/pseries/setup.c | 164 +++ drivers/pci/pcie/aer/aerdrv_core.c | 3 + include/linux/pci.h | 36 ++ 13 files changed, 605 insertions(+), 73 deletions(-) -- 2.14.3 (Apple Git-98)
[PATCH v2 7/7] pseries/setup: Add Initialization of VF Bars
When enabling SR-IOV in pseries platform,
the VF bar properties for a PF are reported on
the device node in the device tree.
This patch adds the IOV Bar resources to Linux
structures from the device tree for later use
when configuring SR-IOV by PF driver.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci.h | 2 +
arch/powerpc/kernel/pci_of_scan.c | 2 +-
arch/powerpc/platforms/pseries/setup.c | 167 +
3 files changed, 170 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 8dc32eacc97c..d82802ff5088 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -121,6 +121,8 @@ extern int remove_phb_dynamic(struct pci_controller *phb);
extern struct pci_dev *of_create_pci_dev(struct device_node *node,
struct pci_bus *bus, int devfn);
+extern unsigned int pci_parse_of_flags(u32 addr0, int bridge);
+
extern void of_scan_pci_bridge(struct pci_dev *dev);
extern void of_scan_bus(struct device_node *node, struct pci_bus *bus);
diff --git a/arch/powerpc/kernel/pci_of_scan.c
b/arch/powerpc/kernel/pci_of_scan.c
index 0d790f8432d2..20ceec4a5f5e 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -38,7 +38,7 @@ static u32 get_int_prop(struct device_node *np, const char
*name, u32 def)
* @addr0: value of 1st cell of a device tree PCI address.
* @bridge: Set this flag if the address is from a bridge 'ranges' property
*/
-static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
+unsigned int pci_parse_of_flags(u32 addr0, int bridge)
{
unsigned int flags = 0;
diff --git a/arch/powerpc/platforms/pseries/setup.c
b/arch/powerpc/platforms/pseries/setup.c
index 5f1beb8367ac..cbb716315070 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -459,6 +459,165 @@ static void __init find_and_init_phbs(void)
of_pci_check_probe_only();
}
+#ifdef CONFIG_PCI_IOV
+enum rtas_iov_fw_value_map {
+ NUM_RES_PROPERTY = 0, ///< Number of Resources
+ LOW_INT = 1, ///< Lowest 32 bits of Address
+ START_OF_ENTRIES = 2, ///< Always start of entry
+ APERTURE_PROPERTY = 2, ///< Start of entry+ to Aperture Size
+ WDW_SIZE_PROPERTY = 4, ///< Start of entry+ to Window Size
+ NEXT_ENTRY= 7 ///< Go to next entry on array
+};
+
+enum get_iov_fw_value_index {
+ BAR_ADDRS = 1,///< Get Bar Address
+ APERTURE_SIZE = 2,///< Get Aperture Size
+ WDW_SIZE = 3 ///< Get Window Size
+};
+
+resource_size_t pseries_get_iov_fw_value(struct pci_dev *dev, int resno,
+enum get_iov_fw_value_index value)
+{
+ const int *indexes;
+ struct device_node *dn = pci_device_to_OF_node(dev);
+ int i, num_res, ret = 0;
+
+ indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
+ if (!indexes)
+ return 0;
+
+ /*
+* First element in the array is the number of Bars
+* returned. Search through the list to find the matching
+* bar
+*/
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ if (resno >= num_res)
+ return 0; /* or an errror */
+
+ i = START_OF_ENTRIES + NEXT_ENTRY * resno;
+ switch (value) {
+ case BAR_ADDRS:
+ ret = of_read_number([i], 2);
+ break;
+ case APERTURE_SIZE:
+ ret = of_read_number([i + APERTURE_PROPERTY], 2);
+ break;
+ case WDW_SIZE:
+ ret = of_read_number([i + WDW_SIZE_PROPERTY], 2);
+ break;
+ }
+
+ return ret;
+}
+
+void of_pci_set_vf_bar_size(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *res;
+ resource_size_t base, size;
+ int i, r, num_res;
+
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ num_res = min_t(int, num_res, PCI_SRIOV_NUM_BARS);
+ for (i = START_OF_ENTRIES, r = 0; r < num_res && r < PCI_SRIOV_NUM_BARS;
+i += NEXT_ENTRY, r++) {
+ res = >resource[r + PCI_IOV_RESOURCES];
+ base = of_read_number([i], 2);
+ size = of_read_number([i + APERTURE_PROPERTY], 2);
+ res->flags = pci_parse_of_flags(of_read_number
+ ([i + LOW_INT], 1), 0);
+ res->flags |= (IORESOURCE_MEM_64 | IORESOURCE_PCI_FIXED);
+ res->name = pci_name(dev);
+ res->start = base;
+ res->end = base + size - 1;
+ }
+}
+
+void of_pci_parse_iov_addrs(struct pci_dev *dev, const int *indexes)
+{
+ struc
[PATCH v2 6/7] pseries/pci: Associate PEs to VFs in configure SR-IOV
After initial validation of SR-IOV resources, firmware will
associate PEs to the dynamic VFs created within this call. This
patch adds the association of PEs to the PF array of PE numbers
indexed by VF.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/pci.c | 151 ++-
1 file changed, 148 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 48d3af026f90..00e6475004ff 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -57,18 +57,163 @@ void pcibios_name_device(struct pci_dev *dev)
}
DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
#endif
-
#ifdef CONFIG_PCI_IOV
+#define MAX_VFS_FOR_MAP_PE 256
+struct pe_map_bar_entry {
+ __be64 bar; /* Input: Virtual Function BAR */
+ __be16 rid; /* Input: Virtual Function Router ID */
+ __be16 pe_num;/* Output: Virtual Function PE Number */
+ __be32 reserved; /* Reserved Space */
+};
+
+int pseries_send_map_pe(struct pci_dev *pdev,
+ u16 num_vfs,
+ struct pe_map_bar_entry *vf_pe_array)
+{
+ struct pci_dn *pdn;
+ int rc;
+ unsigned long buid, addr;
+ int ibm_map_pes = rtas_token("ibm,open-sriov-map-pe-number");
+
+ if (ibm_map_pes == RTAS_UNKNOWN_SERVICE)
+ return -EINVAL;
+
+ pdn = pci_get_pdn(pdev);
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array,
+ RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_map_pes, 5, 1, NULL, addr,
+ BUID_HI(buid), BUID_LO(buid),
+ rtas_data_buf,
+ num_vfs * sizeof(struct pe_map_bar_entry));
+ memcpy(vf_pe_array, rtas_data_buf, RTAS_DATA_BUF_SIZE);
+ spin_unlock(_data_buf_lock);
+
+ if (rc)
+ dev_err(>dev,
+ "%s: Failed to associate pes PE#%lx, rc=%x\n",
+ __func__, addr, rc);
+
+ return rc;
+}
+
+void pseries_set_pe_num(struct pci_dev *pdev, u16 vf_index, __be16 pe_num)
+{
+ struct pci_dn *pdn;
+
+ pdn = pci_get_pdn(pdev);
+ pdn->pe_num_map[vf_index] = be16_to_cpu(pe_num);
+ dev_dbg(>dev, "VF %04x:%02x:%02x.%x associated with PE#%x\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number,
+ PCI_SLOT(pci_iov_virtfn_devfn(pdev, vf_index)),
+ PCI_FUNC(pci_iov_virtfn_devfn(pdev, vf_index)),
+ pdn->pe_num_map[vf_index]);
+}
+
+int pseries_associate_pes(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ int i, rc, vf_index;
+ struct pe_map_bar_entry *vf_pe_array;
+ struct resource *res;
+ u64 size;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ pdn = pci_get_pdn(pdev);
+ /* create firmware structure to associate pes */
+ for (vf_index = 0; vf_index < num_vfs; vf_index++) {
+ pdn->pe_num_map[vf_index] = IODA_INVALID_PE;
+ for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
+ res = >resource[i + PCI_IOV_RESOURCES];
+ if (!res->parent)
+ continue;
+ size = pcibios_iov_resource_alignment(pdev, i +
+ PCI_IOV_RESOURCES);
+ vf_pe_array[vf_index].bar =
+ cpu_to_be64(res->start + size * vf_index);
+ vf_pe_array[vf_index].rid =
+ cpu_to_be16((pci_iov_virtfn_bus(pdev, vf_index)
+ << 8) | pci_iov_virtfn_devfn(pdev,
+ vf_index));
+ vf_pe_array[vf_index].pe_num =
+ cpu_to_be16(IODA_INVALID_PE);
+ }
+ }
+
+ rc = pseries_send_map_pe(pdev, num_vfs, vf_pe_array);
+ /* Only zero is success */
+ if (!rc)
+ for (vf_index = 0; vf_index < num_vfs; vf_index++)
+ pseries_set_pe_num(pdev, vf_index,
+ vf_pe_array[vf_index].pe_num);
+
+ kfree(vf_pe_array);
+ return rc;
+}
+
+int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ intrc;
+ const int *max_vfs;
+ int max_config_vfs;
+ struct device_node *dn = pci_device_to_OF_node(pdev);
+
[PATCH v2 1/7] platform/pseries: Update VF config space after EEH
Add EEH platform operations for pseries to update VF
config space. With this change after EEH, the VF
will have updated config space for pseries platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh.c| 59 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 65 ++--
arch/powerpc/platforms/pseries/eeh_pseries.c | 26 ++-
4 files changed, 88 insertions(+), 63 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 5161c37dd039..82829c65f31a 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -297,6 +297,7 @@ int eeh_pe_reset(struct eeh_pe *pe, int option);
int eeh_pe_configure(struct eeh_pe *pe);
int eeh_pe_inject_err(struct eeh_pe *pe, int type, int func,
unsigned long addr, unsigned long mask);
+int eeh_restore_vf_config(struct pci_dn *pdn);
/**
* EEH_POSSIBLE_ERROR() -- test for possible MMIO failure.
diff --git a/arch/powerpc/kernel/eeh.c b/arch/powerpc/kernel/eeh.c
index cbca0a667682..cc649809885e 100644
--- a/arch/powerpc/kernel/eeh.c
+++ b/arch/powerpc/kernel/eeh.c
@@ -740,6 +740,65 @@ static void *eeh_restore_dev_state(void *data, void
*userdata)
return NULL;
}
+int eeh_restore_vf_config(struct pci_dn *pdn)
+{
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+ u32 devctl, cmd, cap2, aer_capctl;
+ int old_mps;
+
+ if (edev->pcie_cap) {
+ /* Restore MPS */
+ old_mps = (ffs(pdn->mps) - 8) << 5;
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
+ devctl |= old_mps;
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+
+ /* Disable Completion Timeout */
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCAP2,
+4, );
+ if (cap2 & 0x10) {
+ eeh_ops->read_config(pdn,
+edev->pcie_cap + PCI_EXP_DEVCTL2,
+4, );
+ cap2 |= 0x10;
+ eeh_ops->write_config(pdn,
+ edev->pcie_cap + PCI_EXP_DEVCTL2,
+ 4, cap2);
+ }
+ }
+
+ /* Enable SERR and parity checking */
+ eeh_ops->read_config(pdn, PCI_COMMAND, 2, );
+ cmd |= (PCI_COMMAND_PARITY | PCI_COMMAND_SERR);
+ eeh_ops->write_config(pdn, PCI_COMMAND, 2, cmd);
+
+ /* Enable report various errors */
+ if (edev->pcie_cap) {
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_CERE;
+ devctl |= (PCI_EXP_DEVCTL_NFERE |
+ PCI_EXP_DEVCTL_FERE |
+ PCI_EXP_DEVCTL_URRE);
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+ }
+
+ /* Enable ECRC generation and check */
+ if (edev->pcie_cap && edev->aer_cap) {
+ eeh_ops->read_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+4, _capctl);
+ aer_capctl |= (PCI_ERR_CAP_ECRC_GENE | PCI_ERR_CAP_ECRC_CHKE);
+ eeh_ops->write_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+ 4, aer_capctl);
+ }
+
+ return 0;
+}
+
/**
* pcibios_set_pcie_reset_state - Set PCI-E reset state
* @dev: pci device struct
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 961e64115d92..0665b6d03cb3 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1655,70 +1655,11 @@ static int pnv_eeh_next_error(struct eeh_pe **pe)
return ret;
}
-static int pnv_eeh_restore_vf_config(struct pci_dn *pdn)
-{
- struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
- u32 devctl, cmd, cap2, aer_capctl;
- int old_mps;
-
- if (edev->pcie_cap) {
- /* Restore MPS */
- old_mps = (ffs(pdn->mps) - 8) << 5;
- eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
-2, );
- devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
- devctl |= old_mps;
- eeh_ops->write_conf
[PATCH v2 5/7] powerpc/kernel: Add EEH notify resume sysfs
Introduce a method for notify resume to be
called from sysfs. In this patch one can
now call notify resume from sysfs when
is supported by platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_sysfs.c | 46 -
1 file changed, 45 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/eeh_sysfs.c b/arch/powerpc/kernel/eeh_sysfs.c
index 797549289798..344e7607f2e0 100644
--- a/arch/powerpc/kernel/eeh_sysfs.c
+++ b/arch/powerpc/kernel/eeh_sysfs.c
@@ -90,6 +90,38 @@ static ssize_t eeh_pe_state_store(struct device *dev,
static DEVICE_ATTR_RW(eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+static ssize_t eeh_notify_resume_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ pdn = pci_get_pdn(pdev);
+ return sprintf(buf, "%d\n", pdn->last_allow_rc);
+}
+
+static ssize_t eeh_notify_resume_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ if (eeh_ops->notify_resume(pci_get_pdn(pdev)))
+ return -EIO;
+ return count;
+}
+static DEVICE_ATTR_RW(eeh_notify_resume);
+#endif
+
void eeh_sysfs_add_device(struct pci_dev *pdev)
{
struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
@@ -104,7 +136,13 @@ void eeh_sysfs_add_device(struct pci_dev *pdev)
rc += device_create_file(>dev, _attr_eeh_mode);
rc += device_create_file(>dev, _attr_eeh_pe_config_addr);
rc += device_create_file(>dev, _attr_eeh_pe_state);
-
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ rc += device_create_file(>dev,
+_attr_eeh_notify_resume);
+#endif
if (rc)
pr_warn("EEH: Unable to create sysfs entries\n");
else if (edev)
@@ -128,6 +166,12 @@ void eeh_sysfs_remove_device(struct pci_dev *pdev)
device_remove_file(>dev, _attr_eeh_mode);
device_remove_file(>dev, _attr_eeh_pe_config_addr);
device_remove_file(>dev, _attr_eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ device_remove_file(>dev, _attr_eeh_notify_resume);
+#endif
if (edev)
edev->mode &= ~EEH_DEV_SYSFS;
--
2.14.3 (Apple Git-98)
[PATCH v2 2/7] powerpc/kernel: Add uevents in EEH error/resume
Devices can go offline when EEH is reported. This patch adds
a change to the kernel object and lets udev know of error.
When device resumes a change is also set reporting device as
online. Therefore, EEH events are better propagated to user
space for devices in powerpc arch.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_driver.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3c0fa99c5533..9d4e8177c2e0 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -204,6 +204,7 @@ static void *eeh_report_error(void *data, void *userdata)
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
+ char *envp[] = {"EVENT=EEH_ERROR", "ONLINE=0", NULL};
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -228,6 +229,7 @@ static void *eeh_report_error(void *data, void *userdata)
edev->in_error = true;
eeh_pcid_put(dev);
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
return NULL;
}
@@ -358,6 +360,7 @@ static void *eeh_report_resume(void *data, void *userdata)
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
bool was_in_error;
struct pci_driver *driver;
+ char *envp[] = {"EVENT=EEH_RESUME", "ONLINE=1", NULL};
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -381,6 +384,7 @@ static void *eeh_report_resume(void *data, void *userdata)
driver->err_handler->resume(dev);
eeh_pcid_put(dev);
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
return NULL;
}
@@ -397,6 +401,7 @@ static void *eeh_report_failure(void *data, void *userdata)
struct eeh_dev *edev = (struct eeh_dev *)data;
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
struct pci_driver *driver;
+ char * envp[] = {"EVENT=EEH_PERMANENT_FAILURE", "ONLINE=0", NULL};
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -415,6 +420,7 @@ static void *eeh_report_failure(void *data, void *userdata)
driver->err_handler->error_detected(dev, pci_channel_io_perm_failure);
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
eeh_pcid_put(dev);
return NULL;
}
--
2.14.3 (Apple Git-98)
[PATCH v2 3/7] platforms/pseries: Set eeh_pe of EEH_PE_VF type
To correctly use EEH code one has to make
sure that the EEH_PE_VF is set for dynamic created
VFs. Therefore this patch allocates an eeh_pe of
eeh type EEH_PE_VF and associates PE with parent.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci-bridge.h| 5 -
arch/powerpc/platforms/pseries/eeh_pseries.c | 13 -
2 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 9f66ddebb799..16d70740a76f 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -211,7 +211,10 @@ struct pci_dn {
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
boolm64_single_mode;/* Use M64 BAR in Single Mode */
#define IODA_INVALID_M64(-1)
- int (*m64_map)[PCI_SRIOV_NUM_BARS];
+ union {
+ int (*m64_map)[PCI_SRIOV_NUM_BARS]; /*Only used in powernv
*/
+ int last_allow_rc; /* Only used in pSeries */
+ };
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
struct list_head child_list;
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index a671ef4f57f5..c4bc5dd0c6ec 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -58,6 +58,8 @@ static int ibm_configure_pe;
void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
{
struct pci_dn *pdn = pci_get_pdn(pdev);
+ struct pci_dn *physfn_pdn;
+ struct eeh_dev *edev;
if (!pdev->is_virtfn)
return;
@@ -65,6 +67,14 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
pdn->device_id = pdev->device;
pdn->vendor_id = pdev->vendor;
pdn->class_code = pdev->class;
+ /* Last allow unfreeze return code used for retrieval
+* by user space in eeh-sysfs to show the last command
+* completion from platform
+*/
+ pdn->last_allow_rc = 0;
+ physfn_pdn = pci_get_pdn(pdev->physfn);
+ pdn->pe_number = physfn_pdn->pe_num_map[pdn->vf_index];
+ edev = pdn_to_eeh_dev(pdn);
/*
* The following operations will fail if VF's sysfs files
@@ -72,8 +82,9 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
*/
eeh_add_device_early(pdn);
eeh_add_device_late(pdev);
+ edev->pe_config_addr = (pdn->busno << 16) | (pdn->devfn << 8);
+ eeh_add_to_parent_pe(edev);
eeh_sysfs_add_device(pdev);
-
}
/*
--
2.14.3 (Apple Git-98)
[PATCH v2 4/7] powerpc/kernel Add EEH operations to notify resume
When pseries SR-IOV is enabled and after a PF driver
has resumed from EEH, platform has to be notified
of the event so the child VFs can be allowed to
resume their normal recovery path.
This patch makes the EEH operation allow unfreeze
platform dependent code and adds the call to
pseries EEH code.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh_driver.c | 3 +
arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +-
arch/powerpc/platforms/pseries/eeh_pseries.c | 90 +++-
4 files changed, 95 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 82829c65f31a..fd37cc101f4f 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -214,6 +214,7 @@ struct eeh_ops {
int (*write_config)(struct pci_dn *pdn, int where, int size, u32 val);
int (*next_error)(struct eeh_pe **pe);
int (*restore_config)(struct pci_dn *pdn);
+ int (*notify_resume)(struct pci_dn *pdn);
};
extern int eeh_subsystem_flags;
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 9d4e8177c2e0..ed710076b042 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -385,6 +385,9 @@ static void *eeh_report_resume(void *data, void *userdata)
eeh_pcid_put(dev);
kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
+#ifdef CONFIG_PCI_IOV
+ eeh_ops->notify_resume(eeh_dev_to_pdn(edev));
+#endif
return NULL;
}
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 0665b6d03cb3..33c86c1a1720 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1704,7 +1704,8 @@ static struct eeh_ops pnv_eeh_ops = {
.read_config= pnv_eeh_read_config,
.write_config = pnv_eeh_write_config,
.next_error = pnv_eeh_next_error,
- .restore_config = pnv_eeh_restore_config
+ .restore_config = pnv_eeh_restore_config,
+ .notify_resume = NULL
};
#ifdef CONFIG_PCI_IOV
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index c4bc5dd0c6ec..1bae44f30b73 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -743,6 +743,93 @@ static int pseries_eeh_restore_config(struct pci_dn *pdn)
return ret;
}
+#ifdef CONFIG_PCI_IOV
+int pseries_send_allow_unfreeze(struct pci_dn *pdn,
+ u16 *vf_pe_array, int cur_vfs)
+{
+ int rc;
+ int ibm_allow_unfreeze = rtas_token("ibm,open-sriov-allow-unfreeze");
+ unsigned long buid, addr;
+
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array, RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_allow_unfreeze, 5, 1, NULL,
+ addr,
+ BUID_HI(buid),
+ BUID_LO(buid),
+ rtas_data_buf, cur_vfs * sizeof(u16));
+ spin_unlock(_data_buf_lock);
+ if (rc)
+ pr_warn("%s: Failed to allow unfreeze for PHB#%x-PE#%lx,
rc=%x\n",
+ __func__,
+ pdn->phb->global_number, addr, rc);
+ return rc;
+}
+
+static int pseries_call_allow_unfreeze(struct eeh_dev *edev)
+{
+ struct pci_dn *pdn, *tmp, *parent, *physfn_pdn;
+ int cur_vfs = 0, rc = 0, vf_index, bus, devfn;
+ u16 *vf_pe_array;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ if (edev->pdev->is_physfn) {
+ cur_vfs = pci_num_vf(edev->pdev);
+ pdn = eeh_dev_to_pdn(edev);
+ parent = pdn->parent;
+
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++)
+ vf_pe_array[vf_index] =
+ cpu_to_be16(pdn->pe_num_map[vf_index]);
+ rc = pseries_send_allow_unfreeze(pdn, vf_pe_array, cur_vfs);
+ pdn->last_allow_rc = rc;
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++) {
+ list_for_each_entry_safe(pdn, tmp, >child_list,
+list) {
+ bus = pci_iov_virtfn_bus(edev->pdev, vf_index);
+ devfn = pci_iov_virtfn_devfn(edev->pdev,
+vf_index);
+
[PATCH v1 0/7] SR-IOV Enablement on PowerVM
This patch series will enable SR-IOV on PowerVM. A specific set of lids for PFW/PHYP is required. They are planned to release with 920 at the moment. For IBM internal testers let me know of a system you want to test on and we can put on the lids required or we can provide a system to run the tests. This patch depends on the three patches: 988fc3ba5653278a8c14d6ccf687371775930d2b dae7253f9f78a731755ca20c66b2d2c40b86baea 608c0d8804ef3ca4cda8ec6ad914e47deb283d7b v1 - Initial Patch v2 - Addressed Alexey and Russell's comments Bryant G. Ly (7): platform/pseries: Update VF config space after EEH powerpc/kernel: Add uevents in EEH error/resume platforms/pseries: Set eeh_pe of EEH_PE_VF type powerpc/kernel Add EEH operations to notify resume powerpc/kernel: Add EEH notify resume sysfs pseries/pci: Associate PEs to VFs in configure SR-IOV pseries/setup: Add Initialization of VF Bars arch/powerpc/include/asm/eeh.h | 1 + arch/powerpc/include/asm/pci-bridge.h| 5 +- arch/powerpc/include/asm/pci.h | 2 + arch/powerpc/kernel/eeh_driver.c | 9 +- arch/powerpc/kernel/eeh_sysfs.c | 46 ++- arch/powerpc/kernel/pci_of_scan.c| 2 +- arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +- arch/powerpc/platforms/pseries/eeh_pseries.c | 192 ++- arch/powerpc/platforms/pseries/pci.c | 156 +- arch/powerpc/platforms/pseries/setup.c | 183 + 10 files changed, 589 insertions(+), 10 deletions(-) -- 2.14.3 (Apple Git-98)
Re: [PATCH v1 2/7] powerpc/kernel: Add uevents in EEH error/resume
On 12/17/17 9:54 PM, Alexey Kardashevskiy wrote:
> On 14/12/17 02:32, Bryant G. Ly wrote:
>> Devices can go offline when EEH is reported. This patch adds
>> a change to the kernel object and lets udev know of error.
>> When device resumes a change is also set reporting device as
>> online. Therefore, EEH events are better propagated to user
>> space for devices in powerpc arch.
>>
>> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
>> Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
>> ---
>> arch/powerpc/kernel/eeh_driver.c | 5 -
>> 1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/powerpc/kernel/eeh_driver.c
>> b/arch/powerpc/kernel/eeh_driver.c
>> index 3c0fa99c5533..c61bf770282b 100644
>> --- a/arch/powerpc/kernel/eeh_driver.c
>> +++ b/arch/powerpc/kernel/eeh_driver.c
>> @@ -204,6 +204,7 @@ static void *eeh_report_error(void *data, void *userdata)
>> struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
>> enum pci_ers_result rc, *res = userdata;
>> struct pci_driver *driver;
>> +char *envp[] = {"EVENT=EEH_ERROR", "ONLINE=0", NULL};
> scripts/checkpatch.pl:
>
> WARNING: char * array declaration might be better as static const
> #27: FILE: arch/powerpc/kernel/eeh_driver.c:207:
> + char *envp[] = {"EVENT=EEH_ERROR", "ONLINE=0", NULL};
>
>
>
>>
>> if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
>> return NULL;
>> @@ -228,6 +229,7 @@ static void *eeh_report_error(void *data, void *userdata)
>>
>> edev->in_error = true;
>> eeh_pcid_put(dev);
>> +kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
>> return NULL;
>> }
>>
>> @@ -358,6 +360,7 @@ static void *eeh_report_resume(void *data, void
>> *userdata)
>> struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
>> bool was_in_error;
>> struct pci_driver *driver;
>> +char *envp[] = {"EVENT=EEH_RESUME", "ONLINE=1", NULL};
>
> WARNING: char * array declaration might be better as static const
> #43: FILE: arch/powerpc/kernel/eeh_driver.c:363:
> + char *envp[] = {"EVENT=EEH_RESUME", "ONLINE=1", NULL};
>
>
>
Checkpatch is wrong it doesn't check the function that uses it, which only
takes a char *
-Bryant
[PATCH v1 2/7] powerpc/kernel: Add uevents in EEH error/resume
Devices can go offline when EEH is reported. This patch adds
a change to the kernel object and lets udev know of error.
When device resumes a change is also set reporting device as
online. Therefore, EEH events are better propagated to user
space for devices in powerpc arch.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_driver.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 3c0fa99c5533..c61bf770282b 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -204,6 +204,7 @@ static void *eeh_report_error(void *data, void *userdata)
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
enum pci_ers_result rc, *res = userdata;
struct pci_driver *driver;
+ char *envp[] = {"EVENT=EEH_ERROR", "ONLINE=0", NULL};
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -228,6 +229,7 @@ static void *eeh_report_error(void *data, void *userdata)
edev->in_error = true;
eeh_pcid_put(dev);
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
return NULL;
}
@@ -358,6 +360,7 @@ static void *eeh_report_resume(void *data, void *userdata)
struct pci_dev *dev = eeh_dev_to_pci_dev(edev);
bool was_in_error;
struct pci_driver *driver;
+ char *envp[] = {"EVENT=EEH_RESUME", "ONLINE=1", NULL};
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -379,8 +382,8 @@ static void *eeh_report_resume(void *data, void *userdata)
}
driver->err_handler->resume(dev);
-
eeh_pcid_put(dev);
+ kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
return NULL;
}
--
2.14.3 (Apple Git-98)
[PATCH v1 6/7] pseries/pci: Associate PEs to VFs in configure SR-IOV
After initial validation of SR-IOV resources, firmware will
associate PEs to the dynamic VFs created within this call. This
patch adds the association of PEs to the PF array of PE numbers
indexed by VF.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/pci.c | 156 ++-
1 file changed, 153 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 48d3af026f90..c90e7d1247a8 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -57,18 +57,168 @@ void pcibios_name_device(struct pci_dev *dev)
}
DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
#endif
-
#ifdef CONFIG_PCI_IOV
+#define MAX_VFS_FOR_MAP_PE 256
+struct pe_map_bar_entry {
+ __be64 bar; ///< Input: Virtual Function BAR
+ __be16 rid; ///< Input: Virtual Function Router ID
+ __be16 pe_num;///< Output: Virtual Function PE Number
+ __be32 reserved; ///< Reserved Space
+};
+
+int pseries_send_map_pe(struct pci_dev *pdev,
+ u16 num_vfs,
+ struct pe_map_bar_entry *vf_pe_array)
+{
+ struct pci_dn *pdn;
+ int rc;
+ unsigned long buid, addr;
+ int ibm_map_pes = rtas_token("ibm,open-sriov-map-pe-number");
+
+ if (ibm_map_pes == RTAS_UNKNOWN_SERVICE)
+ return -EINVAL;
+
+ pdn = pci_get_pdn(pdev);
+ addr = rtas_config_addr(pdn->busno, pdn->devfn, 0);
+ buid = pdn->phb->buid;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array,
+ RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_map_pes, 5, 1, NULL, addr,
+ BUID_HI(buid), BUID_LO(buid),
+ rtas_data_buf,
+ num_vfs * sizeof(struct pe_map_bar_entry));
+ memcpy(vf_pe_array, rtas_data_buf,
+ RTAS_DATA_BUF_SIZE);
+ spin_unlock(_data_buf_lock);
+
+ if (rc)
+ dev_err(>dev,
+ "%s: Failed to associate pes PE#%lx, rc=%x\n",
+ __func__, addr, rc);
+
+ return rc;
+}
+
+void pseries_set_pe_num(struct pci_dev *pdev,
+ u16 vf_index, __be16 pe_num)
+{
+ struct pci_dn *pdn;
+
+ pdn = pci_get_pdn(pdev);
+ pdn->pe_num_map[vf_index] = be16_to_cpu(pe_num);
+ dev_dbg(>dev, "VF %04x:%02x:%02x.%x associated with PE#%x\n",
+ pci_domain_nr(pdev->bus),
+ pdev->bus->number,
+ PCI_SLOT(pci_iov_virtfn_devfn(pdev, vf_index)),
+ PCI_FUNC(pci_iov_virtfn_devfn(pdev, vf_index)),
+ pdn->pe_num_map[vf_index]);
+}
+
+int pseries_associate_pes(struct pci_dev *pdev, u16 num_vfs)
+{
+ struct pci_dn *pdn;
+ int i, rc, vf_index;
+ struct pe_map_bar_entry *vf_pe_array;
+ struct resource *res;
+ u64 size;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ memset(vf_pe_array, 0, RTAS_DATA_BUF_SIZE);
+ pdn = pci_get_pdn(pdev);
+ /* create firmware structure to associate pes */
+ for (vf_index = 0; vf_index < num_vfs && vf_index < MAX_VFS_FOR_MAP_PE;
+vf_index++) {
+ pdn->pe_num_map[vf_index] = IODA_INVALID_PE;
+ for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
+ res = >resource[i + PCI_IOV_RESOURCES];
+ if (!res->parent)
+ continue;
+ size = pcibios_iov_resource_alignment(pdev, i +
+ PCI_IOV_RESOURCES
+ );
+ vf_pe_array[vf_index].bar =
+ be64_to_cpu(res->start + size * vf_index);
+ vf_pe_array[vf_index].rid =
+ be16_to_cpu((pci_iov_virtfn_bus(pdev, vf_index)
+ << 8) | pci_iov_virtfn_devfn(pdev,
+ vf_index));
+ vf_pe_array[vf_index].pe_num =
+ be16_to_cpu(IODA_INVALID_PE);
+ }
+ }
+
+ rc = pseries_send_map_pe(pdev, num_vfs, vf_pe_array);
+ /* Only zero is success */
+ if (!rc)
+ for (vf_index = 0; vf_index < num_vfs && vf_index <
+MAX_VFS_FOR_MAP_PE; vf_index++)
+ pseries_set_pe_num(pdev, vf_index,
+ vf_pe_array[vf_index].pe_num);
+
+
[PATCH v1 7/7] pseries/setup: Add Initialization of VF Bars
When enabling SR-IOV in pseries platform,
the VF bar properties for a PF are reported on
the device node in the device tree.
This patch adds the IOV Bar resources to Linux
structures from the device tree for later use
when configuring SR-IOV by PF driver.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci.h | 2 +
arch/powerpc/kernel/pci_of_scan.c | 2 +-
arch/powerpc/platforms/pseries/setup.c | 183 +
3 files changed, 186 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/pci.h b/arch/powerpc/include/asm/pci.h
index 8dc32eacc97c..d82802ff5088 100644
--- a/arch/powerpc/include/asm/pci.h
+++ b/arch/powerpc/include/asm/pci.h
@@ -121,6 +121,8 @@ extern int remove_phb_dynamic(struct pci_controller *phb);
extern struct pci_dev *of_create_pci_dev(struct device_node *node,
struct pci_bus *bus, int devfn);
+extern unsigned int pci_parse_of_flags(u32 addr0, int bridge);
+
extern void of_scan_pci_bridge(struct pci_dev *dev);
extern void of_scan_bus(struct device_node *node, struct pci_bus *bus);
diff --git a/arch/powerpc/kernel/pci_of_scan.c
b/arch/powerpc/kernel/pci_of_scan.c
index 0d790f8432d2..20ceec4a5f5e 100644
--- a/arch/powerpc/kernel/pci_of_scan.c
+++ b/arch/powerpc/kernel/pci_of_scan.c
@@ -38,7 +38,7 @@ static u32 get_int_prop(struct device_node *np, const char
*name, u32 def)
* @addr0: value of 1st cell of a device tree PCI address.
* @bridge: Set this flag if the address is from a bridge 'ranges' property
*/
-static unsigned int pci_parse_of_flags(u32 addr0, int bridge)
+unsigned int pci_parse_of_flags(u32 addr0, int bridge)
{
unsigned int flags = 0;
diff --git a/arch/powerpc/platforms/pseries/setup.c
b/arch/powerpc/platforms/pseries/setup.c
index 5f1beb8367ac..ce28882cbde8 100644
--- a/arch/powerpc/platforms/pseries/setup.c
+++ b/arch/powerpc/platforms/pseries/setup.c
@@ -459,6 +459,181 @@ static void __init find_and_init_phbs(void)
of_pci_check_probe_only();
}
+#ifdef CONFIG_PCI_IOV
+enum rtas_iov_fw_value_map {
+ NUM_RES_PROPERTY = 0, ///< Number of Resources
+ LOW_INT = 1, ///< Lowest 32 bits of Address
+ START_OF_ENTRIES = 2, ///< Always start of entry
+ APERTURE_PROPERTY = 2, ///< Start of entry+ to Aperture Size
+ WDW_SIZE_PROPERTY = 4, ///< Start of entry+ to Window Size
+ NEXT_ENTRY= 7 ///< Go to next entry on array
+};
+
+enum get_iov_fw_value_index {
+ BAR_ADDRS = 1,///< Get Bar Address
+ APERTURE_SIZE = 2,///< Get Aperture Size
+ WDW_SIZE = 3 ///< Get Window Size
+};
+
+resource_size_t pseries_get_iov_fw_values(struct pci_dev *dev, int resno,
+ enum get_iov_fw_value_index value)
+{
+ struct vf_bar_wdw {
+ __be64 addr;
+ __be64 aperture_size;
+ __be64 wdw_size;
+ };
+
+ struct vf_bar_wdw window_avail[PCI_SRIOV_NUM_BARS];
+ const int *indexes;
+ struct device_node *dn = pci_device_to_OF_node(dev);
+ int i, r, num_res;
+ resource_size_t return_value;
+
+ indexes = of_get_property(dn, "ibm,open-sriov-vf-bar-info", NULL);
+ if (!indexes)
+ return 0;
+
+ memset(window_avail,
+ 0, sizeof(struct vf_bar_wdw) * PCI_SRIOV_NUM_BARS);
+ return_value = 0;
+ /*
+* First element in the array is the number of Bars
+* returned. Search through the list to find the matching
+* bar
+*/
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ for (i = START_OF_ENTRIES, r = 0; r < num_res && r < PCI_SRIOV_NUM_BARS;
+i += NEXT_ENTRY, r++) {
+ window_avail[r].addr = of_read_number([i], 2);
+ window_avail[r].aperture_size =
+ of_read_number([i + APERTURE_PROPERTY], 2);
+ window_avail[r].wdw_size =
+ of_read_number([i + WDW_SIZE_PROPERTY], 2);
+ }
+
+ switch (value) {
+ case BAR_ADDRS:
+ return_value = window_avail[resno].addr;
+ break;
+ case APERTURE_SIZE:
+ return_value = window_avail[resno].aperture_size;
+ break;
+ case WDW_SIZE:
+ return_value = window_avail[resno].wdw_size;
+ break;
+ default:
+ break;
+ }
+ return return_value;
+}
+
+void of_pci_parse_vf_bar_size(struct pci_dev *dev, const int *indexes)
+{
+ struct resource *res;
+ resource_size_t base, size;
+ int i, r, num_res;
+
+ num_res = of_read_number([NUM_RES_PROPERTY], 1);
+ for (i = START_OF_ENTRIES, r = 0; r < num_res && r < PCI_SRIOV_NUM
[PATCH v1 4/7] powerpc/kernel Add EEH operations to notify resume
When pseries SR-IOV is enabled and after a PF driver
has resumed from EEH, platform has to be notified
of the event so the child VFs can be allowed to
resume their normal recovery path.
This patch makes the EEH operation allow unfreeze
platform dependent code and adds the call to
pseries EEH code.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/eeh.h | 1 +
arch/powerpc/kernel/eeh_driver.c | 4 ++
arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +-
arch/powerpc/platforms/pseries/eeh_pseries.c | 100 ++-
4 files changed, 106 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/eeh.h b/arch/powerpc/include/asm/eeh.h
index 5161c37dd039..12d52a0cd447 100644
--- a/arch/powerpc/include/asm/eeh.h
+++ b/arch/powerpc/include/asm/eeh.h
@@ -214,6 +214,7 @@ struct eeh_ops {
int (*write_config)(struct pci_dn *pdn, int where, int size, u32 val);
int (*next_error)(struct eeh_pe **pe);
int (*restore_config)(struct pci_dn *pdn);
+ int (*notify_resume)(struct pci_dn *pdn);
};
extern int eeh_subsystem_flags;
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index c61bf770282b..dbda0cda559b 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -361,6 +361,7 @@ static void *eeh_report_resume(void *data, void *userdata)
bool was_in_error;
struct pci_driver *driver;
char *envp[] = {"EVENT=EEH_RESUME", "ONLINE=1", NULL};
+ struct pci_dn *pdn = eeh_dev_to_pdn(edev);
if (!dev || eeh_dev_removed(edev) || eeh_pe_passed(edev->pe))
return NULL;
@@ -384,6 +385,9 @@ static void *eeh_report_resume(void *data, void *userdata)
driver->err_handler->resume(dev);
eeh_pcid_put(dev);
kobject_uevent_env(>dev.kobj, KOBJ_CHANGE, envp);
+#ifdef CONFIG_PCI_IOV
+ eeh_ops->notify_resume(pdn);
+#endif
return NULL;
}
diff --git a/arch/powerpc/platforms/powernv/eeh-powernv.c
b/arch/powerpc/platforms/powernv/eeh-powernv.c
index 961e64115d92..8575b3a29e7c 100644
--- a/arch/powerpc/platforms/powernv/eeh-powernv.c
+++ b/arch/powerpc/platforms/powernv/eeh-powernv.c
@@ -1763,7 +1763,8 @@ static struct eeh_ops pnv_eeh_ops = {
.read_config= pnv_eeh_read_config,
.write_config = pnv_eeh_write_config,
.next_error = pnv_eeh_next_error,
- .restore_config = pnv_eeh_restore_config
+ .restore_config = pnv_eeh_restore_config,
+ .notify_resume = NULL
};
#ifdef CONFIG_PCI_IOV
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 5bdd1678a9ff..2b36fbf4ce74 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -798,6 +798,103 @@ static int pseries_eeh_restore_config(struct pci_dn *pdn)
return 0;
}
+#ifdef CONFIG_PCI_IOV
+int pseries_send_allow_unfreeze(struct eeh_pe *pe,
+ u16 *vf_pe_array, int cur_vfs)
+{
+ int rc, config_addr;
+ int ibm_allow_unfreeze = rtas_token("ibm,open-sriov-allow-unfreeze");
+
+ config_addr = pe->config_addr;
+ spin_lock(_data_buf_lock);
+ memcpy(rtas_data_buf, vf_pe_array, RTAS_DATA_BUF_SIZE);
+ rc = rtas_call(ibm_allow_unfreeze, 5, 1, NULL,
+ config_addr,
+ BUID_HI(pe->phb->buid),
+ BUID_LO(pe->phb->buid),
+ rtas_data_buf, cur_vfs * sizeof(u16));
+ spin_unlock(_data_buf_lock);
+ if (rc)
+ pr_warn("%s: Failed to allow unfreeze for PHB#%x-PE#%x,
rc=%x\n",
+ __func__,
+ pe->phb->global_number,
+ pe->config_addr, rc);
+ return rc;
+}
+
+static int pseries_call_allow_unfreeze(struct eeh_dev *edev)
+{
+ struct eeh_pe *pe;
+ struct pci_dn *pdn, *tmp, *parent, *physfn_pdn;
+ int cur_vfs, rc, vf_index;
+ u16 *vf_pe_array;
+
+ vf_pe_array = kzalloc(RTAS_DATA_BUF_SIZE, GFP_KERNEL);
+ if (!vf_pe_array)
+ return -ENOMEM;
+
+ memset(vf_pe_array, 0, RTAS_DATA_BUF_SIZE);
+ cur_vfs = 0;
+ rc = 0;
+ if (edev->pdev->is_physfn) {
+ pe = eeh_dev_to_pe(edev);
+ cur_vfs = pci_num_vf(edev->pdev);
+ pdn = eeh_dev_to_pdn(edev);
+ parent = pdn->parent;
+ /* For each of its VF
+* call allow unfreeze
+*/
+ for (vf_index = 0; vf_index < cur_vfs; vf_index++)
+ vf_pe_array[vf_index] =
+ be16_to_c
[PATCH v1 5/7] powerpc/kernel: Add EEH notify resume sysfs
Introduce a method for notify resume to be
called from sysfs. In this patch one can
now call notify resume from sysfs when
is supported by platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/kernel/eeh_sysfs.c | 46 -
1 file changed, 45 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/eeh_sysfs.c b/arch/powerpc/kernel/eeh_sysfs.c
index 797549289798..344e7607f2e0 100644
--- a/arch/powerpc/kernel/eeh_sysfs.c
+++ b/arch/powerpc/kernel/eeh_sysfs.c
@@ -90,6 +90,38 @@ static ssize_t eeh_pe_state_store(struct device *dev,
static DEVICE_ATTR_RW(eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+static ssize_t eeh_notify_resume_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ pdn = pci_get_pdn(pdev);
+ return sprintf(buf, "%d\n", pdn->last_allow_rc);
+}
+
+static ssize_t eeh_notify_resume_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
+
+ if (!edev || !edev->pe)
+ return -ENODEV;
+
+ if (eeh_ops->notify_resume(pci_get_pdn(pdev)))
+ return -EIO;
+ return count;
+}
+static DEVICE_ATTR_RW(eeh_notify_resume);
+#endif
+
void eeh_sysfs_add_device(struct pci_dev *pdev)
{
struct eeh_dev *edev = pci_dev_to_eeh_dev(pdev);
@@ -104,7 +136,13 @@ void eeh_sysfs_add_device(struct pci_dev *pdev)
rc += device_create_file(>dev, _attr_eeh_mode);
rc += device_create_file(>dev, _attr_eeh_pe_config_addr);
rc += device_create_file(>dev, _attr_eeh_pe_state);
-
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ rc += device_create_file(>dev,
+_attr_eeh_notify_resume);
+#endif
if (rc)
pr_warn("EEH: Unable to create sysfs entries\n");
else if (edev)
@@ -128,6 +166,12 @@ void eeh_sysfs_remove_device(struct pci_dev *pdev)
device_remove_file(>dev, _attr_eeh_mode);
device_remove_file(>dev, _attr_eeh_pe_config_addr);
device_remove_file(>dev, _attr_eeh_pe_state);
+#ifdef CONFIG_PCI_IOV
+ if (of_get_property(pci_device_to_OF_node
+ ((pdev->is_physfn ? pdev : pdev->physfn)),
+ "ibm,is-open-sriov-pf", NULL))
+ device_remove_file(>dev, _attr_eeh_notify_resume);
+#endif
if (edev)
edev->mode &= ~EEH_DEV_SYSFS;
--
2.14.3 (Apple Git-98)
[PATCH v1 3/7] platforms/pseries: Set eeh_pe of EEH_PE_VF type
To correctly use EEH code one has to make
sure that the EEH_PE_VF is set for dynamic created
VFs. Therefore this patch allocates an eeh_pe of
eeh type EEH_PE_VF and associates PE with parent.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/pci-bridge.h| 5 -
arch/powerpc/platforms/pseries/eeh_pseries.c | 9 -
2 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 9f66ddebb799..c30c7cba4c30 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -211,7 +211,10 @@ struct pci_dn {
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
boolm64_single_mode;/* Use M64 BAR in Single Mode */
#define IODA_INVALID_M64(-1)
- int (*m64_map)[PCI_SRIOV_NUM_BARS];
+ union {
+ int (*m64_map)[PCI_SRIOV_NUM_BARS];
+ int last_allow_rc;
+ };
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
struct list_head child_list;
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 1a9a6fa91151..5bdd1678a9ff 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -58,6 +58,8 @@ static int ibm_configure_pe;
void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
{
struct pci_dn *pdn = pci_get_pdn(pdev);
+ struct pci_dn *physfn_pdn;
+ struct eeh_dev *edev;
if (!pdev->is_virtfn)
return;
@@ -65,6 +67,10 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
pdn->device_id = pdev->device;
pdn->vendor_id = pdev->vendor;
pdn->class_code = pdev->class;
+ pdn->last_allow_rc = 0;
+ physfn_pdn = pci_get_pdn(pdev->physfn);
+ pdn->pe_number = physfn_pdn->pe_num_map[pdn->vf_index];
+ edev = pdn_to_eeh_dev(pdn);
/*
* The following operations will fail if VF's sysfs files
@@ -72,8 +78,9 @@ void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
*/
eeh_add_device_early(pdn);
eeh_add_device_late(pdev);
+ edev->pe_config_addr = (pdn->busno << 16) | (pdn->devfn << 8);
+ eeh_add_to_parent_pe(edev);
eeh_sysfs_add_device(pdev);
-
}
/*
--
2.14.3 (Apple Git-98)
[PATCH v1 1/7] platform/pseries: Update VF config space after EEH
Add EEH platform operations for pseries to update VF
config space. With this change after EEH, the VF
will have updated config space for pseries platform.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 85 +++-
1 file changed, 84 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 2295f117e2d3..1a9a6fa91151 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -708,6 +708,89 @@ static int pseries_eeh_write_config(struct pci_dn *pdn,
int where, int size, u32
return rtas_write_config(pdn, where, size, val);
}
+static int pseries_eeh_restore_vf_config(struct pci_dn *pdn)
+{
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+ u32 devctl, cmd, cap2, aer_capctl;
+ int old_mps;
+
+ if (edev->pcie_cap) {
+ /* Restore MPS */
+ old_mps = (ffs(pdn->mps) - 8) << 5;
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
+ devctl |= old_mps;
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+
+ /* Disable Completion Timeout */
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCAP2,
+4, );
+ if (cap2 & 0x10) {
+ eeh_ops->read_config(pdn,
+edev->pcie_cap + PCI_EXP_DEVCTL2,
+4, );
+ cap2 |= 0x10;
+ eeh_ops->write_config(pdn,
+ edev->pcie_cap + PCI_EXP_DEVCTL2,
+ 4, cap2);
+ }
+ }
+
+ /* Enable SERR and parity checking */
+ eeh_ops->read_config(pdn, PCI_COMMAND, 2, );
+ cmd |= (PCI_COMMAND_PARITY | PCI_COMMAND_SERR);
+ eeh_ops->write_config(pdn, PCI_COMMAND, 2, cmd);
+
+ /* Enable report various errors */
+ if (edev->pcie_cap) {
+ eeh_ops->read_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+2, );
+ devctl &= ~PCI_EXP_DEVCTL_CERE;
+ devctl |= (PCI_EXP_DEVCTL_NFERE |
+ PCI_EXP_DEVCTL_FERE |
+ PCI_EXP_DEVCTL_URRE);
+ eeh_ops->write_config(pdn, edev->pcie_cap + PCI_EXP_DEVCTL,
+ 2, devctl);
+ }
+
+ /* Enable ECRC generation and check */
+ if (edev->pcie_cap && edev->aer_cap) {
+ eeh_ops->read_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+4, _capctl);
+ aer_capctl |= (PCI_ERR_CAP_ECRC_GENE | PCI_ERR_CAP_ECRC_CHKE);
+ eeh_ops->write_config(pdn, edev->aer_cap + PCI_ERR_CAP,
+ 4, aer_capctl);
+ }
+
+ return 0;
+}
+
+static int pseries_eeh_restore_config(struct pci_dn *pdn)
+{
+ struct eeh_dev *edev = pdn_to_eeh_dev(pdn);
+ s64 ret;
+
+ if (!edev)
+ return -EEXIST;
+
+ /*
+* FIXME: The MPS, error routing rules, timeout setting are worthy
+* to be exported by firmware in extendible way.
+*/
+ if (edev->physfn)
+ ret = pseries_eeh_restore_vf_config(pdn);
+
+ if (ret) {
+ pr_warn("%s: Can't reinit PCI dev 0x%x (%lld)\n",
+ __func__, edev->pe_config_addr, ret);
+ return -EIO;
+ }
+
+ return 0;
+}
+
static struct eeh_ops pseries_eeh_ops = {
.name = "pseries",
.init = pseries_eeh_init,
@@ -723,7 +806,7 @@ static struct eeh_ops pseries_eeh_ops = {
.read_config= pseries_eeh_read_config,
.write_config = pseries_eeh_write_config,
.next_error = NULL,
- .restore_config = NULL
+ .restore_config = pseries_eeh_restore_config
};
/**
--
2.14.3 (Apple Git-98)
[PATCH v1 0/7] SR-IOV Enablement on PowerVM
This patch series will enable SR-IOV on PowerVM. A specific set of lids for PFW/PHYP is required. They are planned to release with 920 at the moment. For IBM internal testers let me know of a system you want to test on and we can put on the lids required or we can provide a system to run the tests. This patch depends on the three patches: 988fc3ba5653278a8c14d6ccf687371775930d2b dae7253f9f78a731755ca20c66b2d2c40b86baea 608c0d8804ef3ca4cda8ec6ad914e47deb283d7b Bryant G. Ly (7): platform/pseries: Update VF config space after EEH powerpc/kernel: Add uevents in EEH error/resume platforms/pseries: Set eeh_pe of EEH_PE_VF type powerpc/kernel Add EEH operations to notify resume powerpc/kernel: Add EEH notify resume sysfs pseries/pci: Associate PEs to VFs in configure SR-IOV pseries/setup: Add Initialization of VF Bars arch/powerpc/include/asm/eeh.h | 1 + arch/powerpc/include/asm/pci-bridge.h| 5 +- arch/powerpc/include/asm/pci.h | 2 + arch/powerpc/kernel/eeh_driver.c | 9 +- arch/powerpc/kernel/eeh_sysfs.c | 46 ++- arch/powerpc/kernel/pci_of_scan.c| 2 +- arch/powerpc/platforms/powernv/eeh-powernv.c | 3 +- arch/powerpc/platforms/pseries/eeh_pseries.c | 192 ++- arch/powerpc/platforms/pseries/pci.c | 156 +- arch/powerpc/platforms/pseries/setup.c | 183 + 10 files changed, 589 insertions(+), 10 deletions(-) -- 2.14.3 (Apple Git-98)
Re: [PATCH v5 0/3] Prepartion for SR-IOV PowerVM Enablement
On 12/8/17 12:03 AM, Alexey Kardashevskiy wrote: > On 06/12/17 02:13, Bryant G. Ly wrote: >> >> On 12/4/17 7:24 PM, Alexey Kardashevskiy wrote: >>> On 05/12/17 02:08, Bryant G. Ly wrote: >>>> On 12/2/17 7:45 PM, Alexey Kardashevskiy wrote: >>>>> On 10/11/17 01:00, Bryant G. Ly wrote: >>>>>> v1 - Initial patch >>>>>> v2 - Addressed Bjorn's comment on creating a highly platform >>>>>> dependent global exported symbol. >>>>>> v3 - Based patch off linux-ppc/master >>>>>> v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request >>>>>> v5 - Fixed comments and commit message >>>>> What is this made against of? I'd like to give it a try but it does not >>>>> apply to Linus'es tree or powerpc/next. Thanks. >>>>> >>>> This was made against powerpc/next back when it was still under 4.14-rc6. >>>> It has been in review for awhile... >>> Sure, sha1 or github tree would be enough though to try. >> 6cff0a118f23b98c604a3604ea9de11338e24fbe >> >> or if you want to just use the github tree its: >> >> https://github.com/powervm/sriov-ppc/tree/upstream > > Thanks! Looks good, works fine (tried MLX5 wit hpowernv - works, pseries > under KVM - prints a nice error instead of crashing), > > Reviewed-by: Alexey Kardashevskiy <a...@ozlabs.ru> > > Thanks for the review/testing. I will drop the full patch that enables SRIOV on PowerVM next week. The patch will require unreleased PHYP and PFW firmware to fully function. I will include you and Russell to those patches. -Bryant > >>>> -Bryant >>>> >>>>>> Bryant G. Ly (3): >>>>>> powerpc/kernel: Separate SR-IOV Calls >>>>>> pseries: Add PSeries SR-IOV Machine dependent calls >>>>>> PCI/IOV: Add pci_vf_drivers_autoprobe() interface >>>>>> >>>>>> arch/powerpc/include/asm/machdep.h | 7 ++ >>>>>> arch/powerpc/include/asm/pci-bridge.h| 4 +--- >>>>>> arch/powerpc/kernel/eeh_driver.c | 4 ++-- >>>>>> arch/powerpc/kernel/pci-common.c | 23 +++ >>>>>> arch/powerpc/kernel/pci_dn.c | 6 - >>>>>> arch/powerpc/platforms/powernv/eeh-powernv.c | 33 >>>>>> ++-- >>>>>> arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- >>>>>> arch/powerpc/platforms/pseries/eeh_pseries.c | 24 >>>>>> arch/powerpc/platforms/pseries/pci.c | 31 >>>>>> ++ >>>>>> drivers/pci/iov.c| 11 ++ >>>>>> include/linux/pci.h | 2 ++ >>>>>> 11 files changed, 118 insertions(+), 33 deletions(-) >>>>>> >
Re: [PATCH v5 0/3] Prepartion for SR-IOV PowerVM Enablement
On 12/4/17 7:24 PM, Alexey Kardashevskiy wrote: > On 05/12/17 02:08, Bryant G. Ly wrote: >> >> On 12/2/17 7:45 PM, Alexey Kardashevskiy wrote: >>> On 10/11/17 01:00, Bryant G. Ly wrote: >>>> v1 - Initial patch >>>> v2 - Addressed Bjorn's comment on creating a highly platform >>>> dependent global exported symbol. >>>> v3 - Based patch off linux-ppc/master >>>> v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request >>>> v5 - Fixed comments and commit message >>> What is this made against of? I'd like to give it a try but it does not >>> apply to Linus'es tree or powerpc/next. Thanks. >>> >> This was made against powerpc/next back when it was still under 4.14-rc6. >> It has been in review for awhile... > Sure, sha1 or github tree would be enough though to try. 6cff0a118f23b98c604a3604ea9de11338e24fbe or if you want to just use the github tree its: https://github.com/powervm/sriov-ppc/tree/upstream > >> -Bryant >> >>>> Bryant G. Ly (3): >>>> powerpc/kernel: Separate SR-IOV Calls >>>> pseries: Add PSeries SR-IOV Machine dependent calls >>>> PCI/IOV: Add pci_vf_drivers_autoprobe() interface >>>> >>>> arch/powerpc/include/asm/machdep.h | 7 ++ >>>> arch/powerpc/include/asm/pci-bridge.h| 4 +--- >>>> arch/powerpc/kernel/eeh_driver.c | 4 ++-- >>>> arch/powerpc/kernel/pci-common.c | 23 +++ >>>> arch/powerpc/kernel/pci_dn.c | 6 - >>>> arch/powerpc/platforms/powernv/eeh-powernv.c | 33 >>>> ++-- >>>> arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- >>>> arch/powerpc/platforms/pseries/eeh_pseries.c | 24 >>>> arch/powerpc/platforms/pseries/pci.c | 31 >>>> ++ >>>> drivers/pci/iov.c| 11 ++ >>>> include/linux/pci.h | 2 ++ >>>> 11 files changed, 118 insertions(+), 33 deletions(-) >>>> >
Re: [PATCH v5 0/3] Prepartion for SR-IOV PowerVM Enablement
On 12/2/17 7:45 PM, Alexey Kardashevskiy wrote: > On 10/11/17 01:00, Bryant G. Ly wrote: >> v1 - Initial patch >> v2 - Addressed Bjorn's comment on creating a highly platform >> dependent global exported symbol. >> v3 - Based patch off linux-ppc/master >> v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request >> v5 - Fixed comments and commit message > > What is this made against of? I'd like to give it a try but it does not > apply to Linus'es tree or powerpc/next. Thanks. > This was made against powerpc/next back when it was still under 4.14-rc6. It has been in review for awhile... -Bryant >> Bryant G. Ly (3): >> powerpc/kernel: Separate SR-IOV Calls >> pseries: Add PSeries SR-IOV Machine dependent calls >> PCI/IOV: Add pci_vf_drivers_autoprobe() interface >> >> arch/powerpc/include/asm/machdep.h | 7 ++ >> arch/powerpc/include/asm/pci-bridge.h| 4 +--- >> arch/powerpc/kernel/eeh_driver.c | 4 ++-- >> arch/powerpc/kernel/pci-common.c | 23 +++ >> arch/powerpc/kernel/pci_dn.c | 6 - >> arch/powerpc/platforms/powernv/eeh-powernv.c | 33 >> ++-- >> arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- >> arch/powerpc/platforms/pseries/eeh_pseries.c | 24 >> arch/powerpc/platforms/pseries/pci.c | 31 ++ >> drivers/pci/iov.c| 11 ++ >> include/linux/pci.h | 2 ++ >> 11 files changed, 118 insertions(+), 33 deletions(-) >> >
Re: [PATCH v5 0/3] Prepartion for SR-IOV PowerVM Enablement
On 11/9/17 8:00 AM, Bryant G. Ly wrote: > v1 - Initial patch > v2 - Addressed Bjorn's comment on creating a highly platform > dependent global exported symbol. > v3 - Based patch off linux-ppc/master > v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request > v5 - Fixed comments and commit message > > Bryant G. Ly (3): > powerpc/kernel: Separate SR-IOV Calls > pseries: Add PSeries SR-IOV Machine dependent calls > PCI/IOV: Add pci_vf_drivers_autoprobe() interface > > arch/powerpc/include/asm/machdep.h | 7 ++ > arch/powerpc/include/asm/pci-bridge.h| 4 +--- > arch/powerpc/kernel/eeh_driver.c | 4 ++-- > arch/powerpc/kernel/pci-common.c | 23 +++ > arch/powerpc/kernel/pci_dn.c | 6 - > arch/powerpc/platforms/powernv/eeh-powernv.c | 33 > ++-- > arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- > arch/powerpc/platforms/pseries/eeh_pseries.c | 24 > arch/powerpc/platforms/pseries/pci.c | 31 ++ > drivers/pci/iov.c| 11 ++ > include/linux/pci.h | 2 ++ > 11 files changed, 118 insertions(+), 33 deletions(-) > Hi Mike, Any updates on when this is going to get pulled? I want to submit the full SR-IOV enablement on PowerVM, but I haven't seen this get merged into 4.15 yet. -Bryant
Re: [PATCH] ibmveth: Kernel crash LSO offload flag toggle
On 11/14/17 8:47 PM, Daniel Axtens wrote:
> Hi Bryant,
>
> This looks a bit better, but...
>
>> The following patch ensures that the bounce_buffer is not null
>> prior to using it within skb_copy_from_linear_data.
> How would this occur?
>
> Looking at ibmveth.c, I see bounce_buffer being freed in ibmveth_close()
> and allocated in ibmveth_open() only. If allocation fails, the whole
> opening of the device fails with -ENOMEM.
>
> It seems your test case - changing TSO - causes ibmveth_set_tso() to
> cause an adaptor restart - an ibmveth_close(dev) and then an
> ibmveth_open(dev). Is this happening in parallel with an out of memory
> condition - is the memory allocation failing?
>
> Alternatively, could it be the case that you're closing the device while
> packets are in flight, and then trying to use a bounce_buffer that's
> been freed elsewhere? Do you need to decouple memory freeing from
> ibmveth_close?
When checksum or largesend attribute for VEA is enabled, it triggers a
ibmveth_set_features->ibmveth_set_csum_offload or ibmveth_set_tso
Those two routines will do a ibmveth_close, which as you see unmaps and frees
the bounce buffer.
Like you said if there is data in transmit, it causes the system to crash due
to bounce_buffer being used when its already freed elsewhere.
This patch just closes the window, bad things can still happen. I wanted to
leave it
up to the people who actively develop in ibmveth to close the window, since
introducing
a lock can be expensive in tx.
>> The problem can be recreated toggling on/off Large send offload.
>>
>> The following script when run (along with some iperf traffic recreates the
>> crash within 5-10 mins or so).
>>
>> while true
>> do
>> ethtool -k ibmveth0 | grep tcp-segmentation-offload
>> ethtool -K ibmveth0 tso off
>> ethtool -k ibmveth0 | grep tcp-segmentation-offload
>> ethtool -K ibmveth0 tso on
>> done
>>
>> Note: This issue happens the very first time largsesend offload is
>> turned off too (but the above script recreates the issue all the times)
>>
>> [76563.914173] Unable to handle kernel paging request for data at address
>> 0x
>> [76563.914197] Faulting instruction address: 0xc0063940
>> [76563.914205] Oops: Kernel access of bad area, sig: 11 [#1]
>> [76563.914210] SMP NR_CPUS=2048 NUMA pSeries
>> [76563.914217] Modules linked in: rpadlpar_io rpaphp dccp_diag dccp tcp_diag
>> udp_diag inet_diag unix_diag af_packet_diag netlink_diag nls_utf8 isofs
>> binfmt_misc pseries_rng rtc_generic autofs4 ibmvfc scsi_transport_fc
>> ibmvscsi ibmveth
>> [76563.914251] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-34-generic
>> #53-Ubuntu
> ^--- yikes!
>
> There are relevant changes to this area since 4.4:
> 2c42bf4b4317 ("ibmveth: check return of skb_linearize in ibmveth_start_xmit")
> 66aa0678efc2 ("ibmveth: Support to enable LSO/CSO for Trunk VEA.")
>
> Does this crash occurs on a more recent kernel?
We DKMS ibmveth, so we have the most recent stuff. Also I have a test 4.14
kernel
that still have this problem.
>
>> diff --git a/drivers/net/ethernet/ibm/ibmveth.c
>> b/drivers/net/ethernet/ibm/ibmveth.c
>> index f210398200ece..1d29b1649118d 100644
>> --- a/drivers/net/ethernet/ibm/ibmveth.c
>> +++ b/drivers/net/ethernet/ibm/ibmveth.c
>> @@ -1092,8 +1092,14 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff
>> *skb,
>> */
>> if (force_bounce || (!skb_is_nonlinear(skb) &&
>> (skb->len < tx_copybreak))) {
>> -skb_copy_from_linear_data(skb, adapter->bounce_buffer,
>> - skb->len);
>> +if (adapter->bounce_buffer) {
>> +skb_copy_from_linear_data(skb, adapter->bounce_buffer,
>> + skb->len);
>> +} else {
>> +adapter->tx_send_failed++;
>> +netdev->stats.tx_dropped++;
>> +goto out;
> Finally, as I alluded to at the top of this message, isn't the
> disappearance of the bounce-buffer a pretty serious issue? As I
> understand it, it means your data structures are now inconsistent. Do
> you need to - at the least - be more chatty here?
>
> Regards,
> Daniel
Yes, we can be more chatty, but like I said previously the real solution is to
make sure
if there is data in transmit to clear that up first before napi_disable in
ibmveth_close.
-Bryant
[PATCH] ibmveth: Kernel crash LSO offload flag toggle
The following patch ensures that the bounce_buffer is not null prior to using it within skb_copy_from_linear_data. The problem can be recreated toggling on/off Large send offload. The following script when run (along with some iperf traffic recreates the crash within 5-10 mins or so). while true do ethtool -k ibmveth0 | grep tcp-segmentation-offload ethtool -K ibmveth0 tso off ethtool -k ibmveth0 | grep tcp-segmentation-offload ethtool -K ibmveth0 tso on done Note: This issue happens the very first time largsesend offload is turned off too (but the above script recreates the issue all the times) [76563.914173] Unable to handle kernel paging request for data at address 0x [76563.914197] Faulting instruction address: 0xc0063940 [76563.914205] Oops: Kernel access of bad area, sig: 11 [#1] [76563.914210] SMP NR_CPUS=2048 NUMA pSeries [76563.914217] Modules linked in: rpadlpar_io rpaphp dccp_diag dccp tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag nls_utf8 isofs binfmt_misc pseries_rng rtc_generic autofs4 ibmvfc scsi_transport_fc ibmvscsi ibmveth [76563.914251] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.4.0-34-generic #53-Ubuntu [76563.914258] task: c000fe9efcc0 ti: c000feab task.ti: c000feab [76563.914265] NIP: c0063940 LR: d0d31788 CTR: c0064100 [76563.914271] REGS: c000feab2ff0 TRAP: 0300 Not tainted (4.4.0-34-generic) [76563.914277] MSR: 80009033 <SF,EE,ME,IR,DR,RI,LE> CR: 4800284e XER: 001a [76563.914294] CFAR: c0008468 DAR: DSISR: 4200 SOFTE: 1 GPR00: f240 c000feab3270 c15b5d00 GPR04: cd9b0004 002a 0006 c000efc0ccac GPR08: d0d3dd28 0080 d0d34758 GPR12: c0064100 ce7f1c80 c000ffdca938 0100 GPR16: c000ffdca738 c000ffdca538 c000feab34a0 c15f4d00 GPR20: c15f4cf0 c000f5945900 GPR24: 8000 c000feab32d0 c000efc0ccac GPR28: c000f23ccb00 c000f5945000 c000f23ccb00 c000efc0cc00 [76563.914380] NIP [c0063940] memcpy_power7+0x40/0x800 [76563.914387] LR [d0d31788] ibmveth_start_xmit+0x1c8/0x8d0 [ibmveth] [76563.914392] Call Trace: [76563.914396] [c000feab3270] [c000feab32d0] 0xc000feab32d0 (unreliable) [76563.914407] [c000feab3360] [c09816f4] dev_hard_start_xmit+0x304/0x530 [76563.914415] [c000feab3440] [c09b6564] sch_direct_xmit+0x124/0x330 [76563.914423] [c000feab34e0] [c0981ddc] __dev_queue_xmit+0x26c/0x770 [76563.914431] [c000feab3580] [c0a1efc0] arp_xmit+0x30/0xd0 [76563.914438] [c000feab35f0] [c0a1f0f4] arp_send_dst.part.0+0x94/0xb0 [76563.914445] [c000feab3660] [c0a1fcb4] arp_solicit+0x114/0x2b0 [76563.914452] [c000feab3730] [c098d8f4] neigh_probe+0x84/0xd0 [76563.914460] [c000feab3760] [c09937cc] neigh_timer_handler+0xbc/0x320 [76563.914468] [c000feab37a0] [c014a3fc] call_timer_fn+0x5c/0x1c0 [76563.914474] [c000feab3830] [c014a8bc] run_timer_softirq+0x31c/0x3f0 [76563.914483] [c000feab3900] [c00bec58] __do_softirq+0x188/0x3e0 [76563.914490] [c000feab39f0] [c00bf128] irq_exit+0xc8/0x100 [76563.914498] [c000feab3a10] [c001f974] timer_interrupt+0xa4/0xe0 [76563.914505] [c000feab3a40] [c0002714] decrementer_common+0x114/0x180 [76563.914515] --- interrupt: 901 at plpar_hcall_norets+0x1c/0x28 [76563.914515] LR = check_and_cede_processor+0x34/0x50 [76563.914525] [c000feab3d30] [c0916bf0] check_and_cede_processor+0x20/0x50 (unreliable) [76563.914534] [c000feab3d90] [c0916e18] shared_cede_loop+0x68/0x170 [76563.914541] [c000feab3dd0] [c0913e20] cpuidle_enter_state+0x160/0x410 [76563.914549] [c000feab3e30] [c0119d48] call_cpuidle+0x78/0xd0 [76563.914556] [c000feab3e70] [c011a11c] cpu_startup_entry+0x37c/0x480 [76563.914564] [c000feab3f30] [c004563c] start_secondary+0x33c/0x360 [76563.914572] [c000feab3f90] [c0008b6c] start_secondary_prolog+0x10/0x14 Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com> Signed-off-by: Sivakumar Krishnasamy <ks...@linux.vnet.ibm.com> Cc: <sta...@vger.kernel.org> # 4.4+ --- drivers/net/ethernet/ibm/ibmveth.c | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/ibm/ibmveth.c b/drivers/net/ethernet/ibm/ibmveth.c index f210398200ece..1d29b1649118d 100644 --- a/drivers/net/ethernet/ibm/ibmveth.c +++ b/drivers/net/ethernet/ibm/ibmveth.c @@ -1092,8 +1092,14 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff *skb, */ if (force_bounce |
Re: [PATCH] ibmveth: Kernel crash LSO offload flag toggle
On 11/13/17 7:07 PM, Daniel Axtens wrote: > Hi Bryant, > > A few things: > > 1) The commit message could probably be trimmed, especially the stack > traces. Yes, I can trim it. > 2) What exactly are you changing and why does it fix the issue? I > couldn't figure that out from the commit message. The bounce buffer can be null when skb_copy_from_linear_data tries to use it and that causes the kernel crash. Thus adding the check to see whether or not bounce buffer is null prior to utilizing it. > > 3) Does this need a Fixes: tag? > Honestly, this bug has existed for a very long time. It only became more evident when commit 66aa0678efc29abd2ab02a09b23f9a8bc9f12a6c was committed. It is rare for a customer to his this issue since a customer only toggles LSO once and leaves it, but to re-create this bug one would need to write a script like the one in the commit message to toggle it on and off in a loop. Its ultimately a legacy bug that was uncovered more recently. >> } >> >> -netdev->min_mtu = IBMVETH_MIN_MTU; >> -netdev->max_mtu = ETH_MAX_MTU; >> - > 4) What does the above hunk do? It seems unrelated... > You are right the above is just cleanup, I should have separated the patch. > Regards, > Daniel >
[PATCH] ibmveth: Kernel crash LSO offload flag toggle
914525] [c000feab3d30] [c0916bf0]
check_and_cede_processor+0x20/0x50 (unreliable)
[76563.914534] [c000feab3d90] [c0916e18] shared_cede_loop+0x68/0x170
[76563.914541] [c000feab3dd0] [c0913e20]
cpuidle_enter_state+0x160/0x410
[76563.914549] [c000feab3e30] [c0119d48] call_cpuidle+0x78/0xd0
[76563.914556] [c000feab3e70] [c011a11c]
cpu_startup_entry+0x37c/0x480
[76563.914564] [c000feab3f30] [c004563c] start_secondary+0x33c/0x360
[76563.914572] [c000feab3f90] [c0008b6c]
start_secondary_prolog+0x10/0x14
[76563.914579] Instruction dump:
[76563.914584] 4185024c 7cc400d0 7cd01120 78c60760 409f0014 8804 38840001
9803
[76563.914596] 38630001 409e0014 a004 38840002 38630002 409d0014
8004
[76563.914613] ---[ end trace 5382b3d78671418e ]---
[76563.916817]
[76565.916870] Kernel panic - not syncing: Fatal exception in interrupt
[76565.919468] ---[ end Kernel panic - not syncing: Fatal exception in interrupt
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Cc: <sta...@vger.kernel.org> # 4.14+
---
drivers/net/ethernet/ibm/ibmveth.c | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/net/ethernet/ibm/ibmveth.c
b/drivers/net/ethernet/ibm/ibmveth.c
index f210398200ece..971ed54332d42 100644
--- a/drivers/net/ethernet/ibm/ibmveth.c
+++ b/drivers/net/ethernet/ibm/ibmveth.c
@@ -1092,8 +1092,14 @@ static netdev_tx_t ibmveth_start_xmit(struct sk_buff
*skb,
*/
if (force_bounce || (!skb_is_nonlinear(skb) &&
(skb->len < tx_copybreak))) {
- skb_copy_from_linear_data(skb, adapter->bounce_buffer,
- skb->len);
+ if (adapter->bounce_buffer) {
+ skb_copy_from_linear_data(skb, adapter->bounce_buffer,
+ skb->len);
+ } else {
+ adapter->tx_send_failed++;
+ netdev->stats.tx_dropped++;
+ goto out;
+ }
descs[0].fields.flags_len = desc_flags | skb->len;
descs[0].fields.address = adapter->bounce_buffer_dma;
@@ -1691,9 +1697,6 @@ static int ibmveth_probe(struct vio_dev *dev, const
struct vio_device_id *id)
netdev->features |= NETIF_F_FRAGLIST;
}
- netdev->min_mtu = IBMVETH_MIN_MTU;
- netdev->max_mtu = ETH_MAX_MTU;
-
memcpy(netdev->dev_addr, mac_addr_p, ETH_ALEN);
if (firmware_has_feature(FW_FEATURE_CMO))
--
2.13.6 (Apple Git-96)
[PATCH v5 3/3] PCI/IOV: Add pci_vf_drivers_autoprobe() interface
Add a pci_vf_drivers_autoprobe() interface. Setting autoprobe to false
on the PF prevents drivers from binding to VFs when they are enabled.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelg...@google.com>
---
arch/powerpc/platforms/pseries/pci.c | 2 ++
drivers/pci/iov.c| 11 +++
include/linux/pci.h | 2 ++
3 files changed, 15 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 14a90cf471cc..48d3af026f90 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -63,6 +63,7 @@ int pseries_pcibios_sriov_enable(struct pci_dev *pdev, u16
num_vfs)
{
/* Allocate PCI data */
add_dev_pci_data(pdev);
+ pci_vf_drivers_autoprobe(pdev, false);
return 0;
}
@@ -70,6 +71,7 @@ int pseries_pcibios_sriov_disable(struct pci_dev *pdev)
{
/* Release PCI data */
remove_dev_pci_data(pdev);
+ pci_vf_drivers_autoprobe(pdev, true);
return 0;
}
#endif
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ac41c8be9200..1440e48094f4 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -634,6 +634,17 @@ void pci_restore_iov_state(struct pci_dev *dev)
}
/**
+ * pci_vf_drivers_autoprobe - set PF property drivers_autoprobe for VFs
+ * @dev: the PCI device
+ * @auto_probe: set VF drivers auto probe flag
+ */
+void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool auto_probe)
+{
+ if (dev->is_physfn)
+ dev->sriov->drivers_autoprobe = auto_probe;
+}
+
+/**
* pci_iov_bus_range - find bus range used by Virtual Function
* @bus: the PCI bus
*
diff --git a/include/linux/pci.h b/include/linux/pci.h
index f4f8ee5a7362..17a08554baa2 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1965,6 +1965,7 @@ int pci_vfs_assigned(struct pci_dev *dev);
int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs);
int pci_sriov_get_totalvfs(struct pci_dev *dev);
resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno);
+void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe);
#else
static inline int pci_iov_virtfn_bus(struct pci_dev *dev, int id)
{
@@ -1992,6 +1993,7 @@ static inline int pci_sriov_get_totalvfs(struct pci_dev
*dev)
{ return 0; }
static inline resource_size_t pci_iov_resource_size(struct pci_dev *dev, int
resno)
{ return 0; }
+static inline void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe) {
}
#endif
#if defined(CONFIG_HOTPLUG_PCI) || defined(CONFIG_HOTPLUG_PCI_MODULE)
--
2.13.6 (Apple Git-96)
[PATCH v5 2/3] pseries: Add PSeries SR-IOV Machine dependent calls
Add calls for PSeries platform to configure/deconfigure
SR-IOV.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 24 +++
arch/powerpc/platforms/pseries/pci.c | 29
2 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 6b812ad990e4..2295f117e2d3 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -55,6 +55,27 @@ static int ibm_get_config_addr_info;
static int ibm_get_config_addr_info2;
static int ibm_configure_pe;
+void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdev->is_virtfn)
+ return;
+
+ pdn->device_id = pdev->device;
+ pdn->vendor_id = pdev->vendor;
+ pdn->class_code = pdev->class;
+
+ /*
+* The following operations will fail if VF's sysfs files
+* aren't created or its resources aren't finalized.
+*/
+ eeh_add_device_early(pdn);
+ eeh_add_device_late(pdev);
+ eeh_sysfs_add_device(pdev);
+
+}
+
/*
* Buffer for reporting slot-error-detail rtas calls. Its here
* in BSS, and not dynamically alloced, so that it ends up in
@@ -120,6 +141,9 @@ static int pseries_eeh_init(void)
/* Set EEH probe mode */
eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
+ /* Set EEH machine dependent code */
+ ppc_md.pcibios_bus_add_device = pseries_pcibios_bus_add_device;
+
return 0;
}
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 09eba5a9929a..14a90cf471cc 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -3,17 +3,17 @@
* Copyright (C) 2003 Anton Blanchard <an...@au.ibm.com>, IBM
*
* pSeries specific routines for PCI.
- *
+ *
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
- *
+ *
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
+ *
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
@@ -54,10 +54,26 @@ void pcibios_name_device(struct pci_dev *dev)
}
}
}
-}
+}
DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
#endif
+#ifdef CONFIG_PCI_IOV
+int pseries_pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ /* Allocate PCI data */
+ add_dev_pci_data(pdev);
+ return 0;
+}
+
+int pseries_pcibios_sriov_disable(struct pci_dev *pdev)
+{
+ /* Release PCI data */
+ remove_dev_pci_data(pdev);
+ return 0;
+}
+#endif
+
static void __init pSeries_request_regions(void)
{
if (!isa_io_base)
@@ -76,6 +92,11 @@ void __init pSeries_final_fixup(void)
pSeries_request_regions();
eeh_addr_cache_build();
+
+#ifdef CONFIG_PCI_IOV
+ ppc_md.pcibios_sriov_enable = pseries_pcibios_sriov_enable;
+ ppc_md.pcibios_sriov_disable = pseries_pcibios_sriov_disable;
+#endif
}
/*
--
2.13.6 (Apple Git-96)
[PATCH v5 0/3] Prepartion for SR-IOV PowerVM Enablement
v1 - Initial patch v2 - Addressed Bjorn's comment on creating a highly platform dependent global exported symbol. v3 - Based patch off linux-ppc/master v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request v5 - Fixed comments and commit message Bryant G. Ly (3): powerpc/kernel: Separate SR-IOV Calls pseries: Add PSeries SR-IOV Machine dependent calls PCI/IOV: Add pci_vf_drivers_autoprobe() interface arch/powerpc/include/asm/machdep.h | 7 ++ arch/powerpc/include/asm/pci-bridge.h| 4 +--- arch/powerpc/kernel/eeh_driver.c | 4 ++-- arch/powerpc/kernel/pci-common.c | 23 +++ arch/powerpc/kernel/pci_dn.c | 6 - arch/powerpc/platforms/powernv/eeh-powernv.c | 33 ++-- arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- arch/powerpc/platforms/pseries/eeh_pseries.c | 24 arch/powerpc/platforms/pseries/pci.c | 31 ++ drivers/pci/iov.c| 11 ++ include/linux/pci.h | 2 ++ 11 files changed, 118 insertions(+), 33 deletions(-) -- 2.13.6 (Apple Git-96)
[PATCH v5 1/3] powerpc/kernel: Separate SR-IOV Calls
SR-IOV can now be enabled in PowerNV platforms and Pseries
platforms. Therefore, the appropriate calls were moved to
machine dependent code instead of definition at compile time.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/include/asm/machdep.h | 7 ++
arch/powerpc/include/asm/pci-bridge.h| 4 +---
arch/powerpc/kernel/eeh_driver.c | 4 ++--
arch/powerpc/kernel/pci-common.c | 23 +++
arch/powerpc/kernel/pci_dn.c | 6 -
arch/powerpc/platforms/powernv/eeh-powernv.c | 33 ++--
arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++--
7 files changed, 54 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/machdep.h
b/arch/powerpc/include/asm/machdep.h
index 73b92017b6d7..20f68d36af8c 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -172,11 +172,18 @@ struct machdep_calls {
/* Called after scan and before resource survey */
void (*pcibios_fixup_phb)(struct pci_controller *hose);
+ /* Called after device has been added to bus and
+* before sysfs has been created
+*/
+ void (*pcibios_bus_add_device)(struct pci_dev *pdev);
+
resource_size_t (*pcibios_default_alignment)(void);
#ifdef CONFIG_PCI_IOV
void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int
resno);
+ int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
+ int (*pcibios_sriov_disable)(struct pci_dev *pdev);
#endif /* CONFIG_PCI_IOV */
/* Called to shutdown machine specific hardware not already controlled
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 0b8aa1fe2d5f..323628ca4d6d 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -203,10 +203,9 @@ struct pci_dn {
struct eeh_dev *edev; /* eeh device */
#endif
#define IODA_INVALID_PE0x
-#ifdef CONFIG_PPC_POWERNV
unsigned int pe_number;
- int vf_index; /* VF index in the PF */
#ifdef CONFIG_PCI_IOV
+ int vf_index; /* VF index in the PF */
u16 vfs_expanded; /* number of VFs IOV BAR expanded */
u16 num_vfs;/* number of VFs enabled*/
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
@@ -215,7 +214,6 @@ struct pci_dn {
int (*m64_map)[PCI_SRIOV_NUM_BARS];
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
-#endif
struct list_head child_list;
struct list_head list;
};
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 8b840191df59..f2d1b369974d 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -440,7 +440,7 @@ static void *eeh_add_virt_device(void *data, void *userdata)
return NULL;
}
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
pci_iov_add_virtfn(edev->physfn, pdn->vf_index, 0);
#endif
return NULL;
@@ -496,7 +496,7 @@ static void *eeh_rmv_device(void *data, void *userdata)
(*removed)++;
if (edev->physfn) {
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
struct pci_dn *pdn = eeh_dev_to_pdn(edev);
pci_iov_remove_virtfn(edev->physfn, pdn->vf_index, 0);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 02831a396419..d45b956d2e3a 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -249,8 +249,31 @@ resource_size_t pcibios_iov_resource_alignment(struct
pci_dev *pdev, int resno)
return pci_iov_resource_size(pdev, resno);
}
+
+int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ if (ppc_md.pcibios_sriov_enable)
+ return ppc_md.pcibios_sriov_enable(pdev, num_vfs);
+
+ return 0;
+}
+
+int pcibios_sriov_disable(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_sriov_disable)
+ return ppc_md.pcibios_sriov_disable(pdev);
+
+ return 0;
+}
+
#endif /* CONFIG_PCI_IOV */
+void pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_bus_add_device)
+ ppc_md.pcibios_bus_add_device(pdev);
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 0e395afbf0f4..ab147a1909c8 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -156,10 +156,8 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn
*parent,
pdn-&g
Re: [PATCH v4 3/3] pci/iov: Expose VF drivers Auto Probe Globally
Add linuxppc-dev@lists.ozlabs.org
On 10/31/17 9:47 AM, Bryant G. Ly wrote:
> When SR-IOV is supported in pci_dev struct the
> member variable drivers_autoprobe controls the
> VF drivers auto probe functionality. This
> patch exposes a gloabal function to be used
> if needed when configuring the pci_dev sriov object.
>
> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
> Signed-off-by: Juan J. Alvarez <jjalv...@linux.vnet.ibm.com>
> ---
> arch/powerpc/platforms/pseries/pci.c | 2 ++
> drivers/pci/iov.c| 12
> include/linux/pci.h | 2 ++
> 3 files changed, 16 insertions(+)
>
> diff --git a/arch/powerpc/platforms/pseries/pci.c
> b/arch/powerpc/platforms/pseries/pci.c
> index 14a90cf471cc..48d3af026f90 100644
> --- a/arch/powerpc/platforms/pseries/pci.c
> +++ b/arch/powerpc/platforms/pseries/pci.c
> @@ -63,6 +63,7 @@ int pseries_pcibios_sriov_enable(struct pci_dev *pdev, u16
> num_vfs)
> {
> /* Allocate PCI data */
> add_dev_pci_data(pdev);
> + pci_vf_drivers_autoprobe(pdev, false);
> return 0;
> }
>
> @@ -70,6 +71,7 @@ int pseries_pcibios_sriov_disable(struct pci_dev *pdev)
> {
> /* Release PCI data */
> remove_dev_pci_data(pdev);
> + pci_vf_drivers_autoprobe(pdev, true);
> return 0;
> }
> #endif
> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
> index ac41c8be9200..41465c06975c 100644
> --- a/drivers/pci/iov.c
> +++ b/drivers/pci/iov.c
> @@ -634,6 +634,18 @@ void pci_restore_iov_state(struct pci_dev *dev)
> }
>
> /**
> + * pci_vf_drivers_autoprobe - set pf property drivers_autoprobe for vfs
> + * @dev: the PCI device
> + * @auto_probe: set vf drivers auto probe flag
> + */
> +void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool auto_probe)
> +{
> + if (dev->is_physfn)
> + dev->sriov->drivers_autoprobe = auto_probe;
> +}
> +EXPORT_SYMBOL_GPL(pci_vf_drivers_autoprobe);
> +
> +/**
> * pci_iov_bus_range - find bus range used by Virtual Function
> * @bus: the PCI bus
> *
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index f4f8ee5a7362..17a08554baa2 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -1965,6 +1965,7 @@ int pci_vfs_assigned(struct pci_dev *dev);
> int pci_sriov_set_totalvfs(struct pci_dev *dev, u16 numvfs);
> int pci_sriov_get_totalvfs(struct pci_dev *dev);
> resource_size_t pci_iov_resource_size(struct pci_dev *dev, int resno);
> +void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe);
> #else
> static inline int pci_iov_virtfn_bus(struct pci_dev *dev, int id)
> {
> @@ -1992,6 +1993,7 @@ static inline int pci_sriov_get_totalvfs(struct pci_dev
> *dev)
> { return 0; }
> static inline resource_size_t pci_iov_resource_size(struct pci_dev *dev, int
> resno)
> { return 0; }
> +static inline void pci_vf_drivers_autoprobe(struct pci_dev *dev, bool probe)
> { }
> #endif
>
> #if defined(CONFIG_HOTPLUG_PCI) || defined(CONFIG_HOTPLUG_PCI_MODULE)
Re: [PATCH v4 2/3] pseries: Add PSeries SR-IOV Machine dependent calls
Add linuxppc-dev@lists.ozlabs.org
On 10/31/17 9:47 AM, Bryant G. Ly wrote:
> Add calls for PSeries platform to configure/deconfigure
> SR-IOV.
>
> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
> Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
> ---
> arch/powerpc/platforms/pseries/eeh_pseries.c | 24 +++
> arch/powerpc/platforms/pseries/pci.c | 29
>
> 2 files changed, 49 insertions(+), 4 deletions(-)
>
> diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
> b/arch/powerpc/platforms/pseries/eeh_pseries.c
> index 6b812ad990e4..2295f117e2d3 100644
> --- a/arch/powerpc/platforms/pseries/eeh_pseries.c
> +++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
> @@ -55,6 +55,27 @@ static int ibm_get_config_addr_info;
> static int ibm_get_config_addr_info2;
> static int ibm_configure_pe;
>
> +void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
> +{
> + struct pci_dn *pdn = pci_get_pdn(pdev);
> +
> + if (!pdev->is_virtfn)
> + return;
> +
> + pdn->device_id = pdev->device;
> + pdn->vendor_id = pdev->vendor;
> + pdn->class_code = pdev->class;
> +
> + /*
> + * The following operations will fail if VF's sysfs files
> + * aren't created or its resources aren't finalized.
> + */
> + eeh_add_device_early(pdn);
> + eeh_add_device_late(pdev);
> + eeh_sysfs_add_device(pdev);
> +
> +}
> +
> /*
> * Buffer for reporting slot-error-detail rtas calls. Its here
> * in BSS, and not dynamically alloced, so that it ends up in
> @@ -120,6 +141,9 @@ static int pseries_eeh_init(void)
> /* Set EEH probe mode */
> eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
>
> + /* Set EEH machine dependent code */
> + ppc_md.pcibios_bus_add_device = pseries_pcibios_bus_add_device;
> +
> return 0;
> }
>
> diff --git a/arch/powerpc/platforms/pseries/pci.c
> b/arch/powerpc/platforms/pseries/pci.c
> index 09eba5a9929a..14a90cf471cc 100644
> --- a/arch/powerpc/platforms/pseries/pci.c
> +++ b/arch/powerpc/platforms/pseries/pci.c
> @@ -3,17 +3,17 @@
> * Copyright (C) 2003 Anton Blanchard <an...@au.ibm.com>, IBM
> *
> * pSeries specific routines for PCI.
> - *
> + *
> * This program is free software; you can redistribute it and/or modify
> * it under the terms of the GNU General Public License as published by
> * the Free Software Foundation; either version 2 of the License, or
> * (at your option) any later version.
> - *
> + *
> * This program is distributed in the hope that it will be useful,
> * but WITHOUT ANY WARRANTY; without even the implied warranty of
> * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> * GNU General Public License for more details.
> - *
> + *
> * You should have received a copy of the GNU General Public License
> * along with this program; if not, write to the Free Software
> * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
> @@ -54,10 +54,26 @@ void pcibios_name_device(struct pci_dev *dev)
> }
> }
> }
> -}
> +}
> DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
> #endif
>
> +#ifdef CONFIG_PCI_IOV
> +int pseries_pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
> +{
> + /* Allocate PCI data */
> + add_dev_pci_data(pdev);
> + return 0;
> +}
> +
> +int pseries_pcibios_sriov_disable(struct pci_dev *pdev)
> +{
> + /* Release PCI data */
> + remove_dev_pci_data(pdev);
> + return 0;
> +}
> +#endif
> +
> static void __init pSeries_request_regions(void)
> {
> if (!isa_io_base)
> @@ -76,6 +92,11 @@ void __init pSeries_final_fixup(void)
> pSeries_request_regions();
>
> eeh_addr_cache_build();
> +
> +#ifdef CONFIG_PCI_IOV
> + ppc_md.pcibios_sriov_enable = pseries_pcibios_sriov_enable;
> + ppc_md.pcibios_sriov_disable = pseries_pcibios_sriov_disable;
> +#endif
> }
>
> /*
Re: [PATCH v4 1/3] powerpc/kernel: Separate SR-IOV Calls
Add linuxppc-dev@lists.ozlabs.org
On 10/31/17 9:47 AM, Bryant G. Ly wrote:
> SR-IOV can now be enabled in PowerNV platforms and Pseries
> platforms. Therefore, the appropriate calls were moved to
> machine dependent code instead of definition at compile time.
>
> Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
> Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
> ---
> arch/powerpc/include/asm/machdep.h | 7 ++
> arch/powerpc/include/asm/pci-bridge.h| 4 +---
> arch/powerpc/kernel/eeh_driver.c | 4 ++--
> arch/powerpc/kernel/pci-common.c | 23 +++
> arch/powerpc/kernel/pci_dn.c | 6 -
> arch/powerpc/platforms/powernv/eeh-powernv.c | 33
> ++--
> arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++--
> 7 files changed, 54 insertions(+), 29 deletions(-)
>
> diff --git a/arch/powerpc/include/asm/machdep.h
> b/arch/powerpc/include/asm/machdep.h
> index 73b92017b6d7..20f68d36af8c 100644
> --- a/arch/powerpc/include/asm/machdep.h
> +++ b/arch/powerpc/include/asm/machdep.h
> @@ -172,11 +172,18 @@ struct machdep_calls {
> /* Called after scan and before resource survey */
> void (*pcibios_fixup_phb)(struct pci_controller *hose);
>
> + /* Called after device has been added to bus and
> + * before sysfs has been created
> + */
> + void (*pcibios_bus_add_device)(struct pci_dev *pdev);
> +
> resource_size_t (*pcibios_default_alignment)(void);
>
> #ifdef CONFIG_PCI_IOV
> void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
> resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int
> resno);
> + int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
> + int (*pcibios_sriov_disable)(struct pci_dev *pdev);
> #endif /* CONFIG_PCI_IOV */
>
> /* Called to shutdown machine specific hardware not already controlled
> diff --git a/arch/powerpc/include/asm/pci-bridge.h
> b/arch/powerpc/include/asm/pci-bridge.h
> index 0b8aa1fe2d5f..323628ca4d6d 100644
> --- a/arch/powerpc/include/asm/pci-bridge.h
> +++ b/arch/powerpc/include/asm/pci-bridge.h
> @@ -203,10 +203,9 @@ struct pci_dn {
> struct eeh_dev *edev; /* eeh device */
> #endif
> #define IODA_INVALID_PE 0x
> -#ifdef CONFIG_PPC_POWERNV
> unsigned int pe_number;
> - int vf_index; /* VF index in the PF */
> #ifdef CONFIG_PCI_IOV
> + int vf_index; /* VF index in the PF */
> u16 vfs_expanded; /* number of VFs IOV BAR expanded */
> u16 num_vfs;/* number of VFs enabled*/
> unsigned int *pe_num_map; /* PE# for the first VF PE or array */
> @@ -215,7 +214,6 @@ struct pci_dn {
> int (*m64_map)[PCI_SRIOV_NUM_BARS];
> #endif /* CONFIG_PCI_IOV */
> int mps;/* Maximum Payload Size */
> -#endif
> struct list_head child_list;
> struct list_head list;
> };
> diff --git a/arch/powerpc/kernel/eeh_driver.c
> b/arch/powerpc/kernel/eeh_driver.c
> index 8b840191df59..f2d1b369974d 100644
> --- a/arch/powerpc/kernel/eeh_driver.c
> +++ b/arch/powerpc/kernel/eeh_driver.c
> @@ -440,7 +440,7 @@ static void *eeh_add_virt_device(void *data, void
> *userdata)
> return NULL;
> }
>
> -#ifdef CONFIG_PPC_POWERNV
> +#ifdef CONFIG_PCI_IOV
> pci_iov_add_virtfn(edev->physfn, pdn->vf_index, 0);
> #endif
> return NULL;
> @@ -496,7 +496,7 @@ static void *eeh_rmv_device(void *data, void *userdata)
> (*removed)++;
>
> if (edev->physfn) {
> -#ifdef CONFIG_PPC_POWERNV
> +#ifdef CONFIG_PCI_IOV
> struct pci_dn *pdn = eeh_dev_to_pdn(edev);
>
> pci_iov_remove_virtfn(edev->physfn, pdn->vf_index, 0);
> diff --git a/arch/powerpc/kernel/pci-common.c
> b/arch/powerpc/kernel/pci-common.c
> index 02831a396419..d45b956d2e3a 100644
> --- a/arch/powerpc/kernel/pci-common.c
> +++ b/arch/powerpc/kernel/pci-common.c
> @@ -249,8 +249,31 @@ resource_size_t pcibios_iov_resource_alignment(struct
> pci_dev *pdev, int resno)
>
> return pci_iov_resource_size(pdev, resno);
> }
> +
> +int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
> +{
> + if (ppc_md.pcibios_sriov_enable)
> + return ppc_md.pcibios_sriov_enable(pdev, num_vfs);
> +
> + return 0;
> +}
> +
> +int pcibios_sriov_disable(struct pci_dev *pdev)
> +{
> + if (ppc_md.pcibios_sriov_disable)
> + return ppc_md.pcibios_sriov_disable(pdev);
> +
> + return 0;
> +}
&
Re: [PATCH v4 0/3] Prepartion for SR-IOV PowerVM Enablement
Add linuxppc-dev@lists.ozlabs.org On 10/31/17 9:47 AM, Bryant G. Ly wrote: > v1 - Initial patch > v2 - Addressed Bjorn's comment on creating a highly platform > dependent global exported symbol. > v3 - Based patch off linux-ppc/master > v4 - Using the sriov-drivers_autoprobe mechanism per Bjorn's request > > Bryant G. Ly (3): > powerpc/kernel: Separate SR-IOV Calls > pseries: Add PSeries SR-IOV Machine dependent calls > pci/iov: Expose VF drivers Auto Probe Globally > > arch/powerpc/include/asm/machdep.h | 7 ++ > arch/powerpc/include/asm/pci-bridge.h| 4 +--- > arch/powerpc/kernel/eeh_driver.c | 4 ++-- > arch/powerpc/kernel/pci-common.c | 23 +++ > arch/powerpc/kernel/pci_dn.c | 6 - > arch/powerpc/platforms/powernv/eeh-powernv.c | 33 > ++-- > arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- > arch/powerpc/platforms/pseries/eeh_pseries.c | 24 > arch/powerpc/platforms/pseries/pci.c | 31 ++ > drivers/pci/iov.c| 12 ++ > include/linux/pci.h | 2 ++ > 11 files changed, 119 insertions(+), 33 deletions(-) >
Re: [PATCH v3 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device
Adding Juan back into the cc: jjalv...@linux.vnet.ibm.com On 10/16/17 10:38 PM, Michael Ellerman wrote: > "Bryant G. Ly" <bryan...@linux.vnet.ibm.com> writes: >> On 10/12/17 1:29 PM, Bjorn Helgaas wrote: > ... >>> If that's the case, how to you ever bind a driver to these VFs? The >>> changelog says you don't want VF drivers to load *immediately*, so I >>> assume you do want them to load eventually. >>> >> The VF's that get dynamically created within the configure SR-IOV call, >> on the Pseries Platform, wont be matched with a driver. - We do not >> want it to match. >> >> The Power Hypervisor will load the VFs. The VF's will get assigned(by >> the user) > via the HMC or Novalink in this environment which will >> then trigger PHYP to load the VF device node to the device tree. > What about the other "Power Hypervisor"? ie. KVM running on Power. This path is only exercised when configuring SR-IOV for Pseries LPAR, therefore it will not affect PowerNV or KVM(opal). Which is the reason for the separation of calls to the machine dependent stuff. > We also use the pseries platform when running under KVM. > > cheers > If anyone plans to enable SR-IOV on Pseries platform, firmware must provide the resources to enable the VFs and map them with system resources. A new version of the PAPR Document will be added to document these system resources. Lastly, we were not aware that there is an intention to enable SR-IOV in adapters assigned to a VM with Pseries running on KVM. Furthermore, this could be left as a todo for the future if this type of configuration is needed. -Bryant
Re: [PATCH v3 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device
On 10/13/17 1:05 PM, Alex Williamson wrote:
On Fri, 13 Oct 2017 07:01:48 -0500
Steven Royer <sero...@linux.vnet.ibm.com> wrote:
On 2017-10-13 06:53, Steven Royer wrote:
On 2017-10-12 22:34, Bjorn Helgaas wrote:
[+cc Alex, Bodong, Eli, Saeed]
On Thu, Oct 12, 2017 at 02:59:23PM -0500, Bryant G. Ly wrote:
On 10/12/17 1:29 PM, Bjorn Helgaas wrote:
On Thu, Oct 12, 2017 at 03:09:53PM +1100, Michael Ellerman wrote:
Bjorn Helgaas <helg...@kernel.org> writes:
On Fri, Sep 22, 2017 at 09:19:28AM -0500, Bryant G. Ly wrote:
This patch adds the machine dependent call for
pcibios_bus_add_device, since the previous patch
separated the calls out between the PowerNV and PowerVM.
The difference here is that for the PowerVM environment
we do not want match_driver set because in this environment
we do not want the VF device drivers to load immediately, due to
firmware loading the device node when VF device is assigned to the
logical partition.
This patch will depend on the patch linked below, which is under
review.
https://patchwork.kernel.org/patch/9882915/
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 24
1 file changed, 24 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 6b812ad990e4..45946ee90985 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -64,6 +64,27 @@ static unsigned char slot_errbuf[RTAS_ERROR_LOG_MAX];
static DEFINE_SPINLOCK(slot_errbuf_lock);
static int eeh_error_buf_size;
+void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdev->is_virtfn)
+ return;
+
+ pdn->device_id = pdev->device;
+ pdn->vendor_id = pdev->vendor;
+ pdn->class_code = pdev->class;
+
+ /*
+* The following operations will fail if VF's sysfs files
+* aren't created or its resources aren't finalized.
+*/
+ eeh_add_device_early(pdn);
+ eeh_add_device_late(pdev);
+ eeh_sysfs_add_device(pdev);
+ pdev->match_driver = -1;
match_driver is a bool, which should be assigned "true" or "false".
Above he mentioned a dependency on:
[04/10] PCI: extend pci device match_driver state
https://patchwork.kernel.org/patch/9882915/
Which makes it an int.
Oh, right, I missed that, thanks.
Or has that patch been rejected or something?
I haven't *rejected* it, but it's low on my priority list, so you
shouldn't depend on it unless it adds functionality you really need.
If I did apply that particular patch, I would want some rework because
it currently obfuscates the match_driver logic. There's no clue when
reading the code what -1/0/1 mean.
So do you prefer enum's? - If so I can make a change for that.
Apparently here you *do* want the "-1 means the PCI core will never
set match_driver to 1" functionality, so maybe you do depend on it.
We depend on the patch because we want that ability to never set
match_driver,
for SRIOV on PowerVM.
Is this really new PowerVM-specific functionality? ISTR recent
discussions
about inhibiting driver binding in a generic way, e.g.,
http://lkml.kernel.org/r/1490022874-54718-1-git-send-email-bod...@mellanox.com
If that's the case, how to you ever bind a driver to these VFs? The
changelog says you don't want VF drivers to load *immediately*, so I
assume you do want them to load eventually.
The VF's that get dynamically created within the configure SR-IOV
call, on the Pseries Platform, wont be matched with a driver. - We
do not want it to match.
The Power Hypervisor will load the VFs. The VF's will get
assigned(by the user) via the HMC or Novalink in this environment
which will then trigger PHYP to load the VF device node to the
device tree.
I don't know what it means for the Hypervisor to "load the VFs." Can
you explain that in PCI-speak?
The things I know about are:
- we set PCI_SRIOV_CTRL_VFE in the PF, which enables VFs
- now the VFs respond to config accesses
- the PCI core enumerates the VFs by reading their config space
- the PCI core builds pci_dev structs for the VFs
- the PCI core adds these pci_devs to the bus
- we try to bind drivers to the VFs
- the VF driver probe function may read VF config space and VF BARs
- the VF may be assigned to a guest VM
Where does "loading the VFs" fit in? I don't know what HMC, Novalink,
or PHYP are. I don't *need* to know what they are, as long as you can
explain what's happening in terms of the PCI concepts and generic
Linux VMs
and device assignment.
Bjorn
The VFs will be hotplugged into the VM separately from the enable
SR-IOV, so the driver will load as part o
Re: [PATCH v3 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device
On 10/12/17 1:29 PM, Bjorn Helgaas wrote:
On Thu, Oct 12, 2017 at 03:09:53PM +1100, Michael Ellerman wrote:
Bjorn Helgaas <helg...@kernel.org> writes:
On Fri, Sep 22, 2017 at 09:19:28AM -0500, Bryant G. Ly wrote:
This patch adds the machine dependent call for
pcibios_bus_add_device, since the previous patch
separated the calls out between the PowerNV and PowerVM.
The difference here is that for the PowerVM environment
we do not want match_driver set because in this environment
we do not want the VF device drivers to load immediately, due to
firmware loading the device node when VF device is assigned to the
logical partition.
This patch will depend on the patch linked below, which is under
review.
https://patchwork.kernel.org/patch/9882915/
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 24
1 file changed, 24 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 6b812ad990e4..45946ee90985 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -64,6 +64,27 @@ static unsigned char slot_errbuf[RTAS_ERROR_LOG_MAX];
static DEFINE_SPINLOCK(slot_errbuf_lock);
static int eeh_error_buf_size;
+void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdev->is_virtfn)
+ return;
+
+ pdn->device_id = pdev->device;
+ pdn->vendor_id = pdev->vendor;
+ pdn->class_code = pdev->class;
+
+ /*
+* The following operations will fail if VF's sysfs files
+* aren't created or its resources aren't finalized.
+*/
+ eeh_add_device_early(pdn);
+ eeh_add_device_late(pdev);
+ eeh_sysfs_add_device(pdev);
+ pdev->match_driver = -1;
match_driver is a bool, which should be assigned "true" or "false".
Above he mentioned a dependency on:
[04/10] PCI: extend pci device match_driver state
https://patchwork.kernel.org/patch/9882915/
Which makes it an int.
Oh, right, I missed that, thanks.
Or has that patch been rejected or something?
I haven't *rejected* it, but it's low on my priority list, so you
shouldn't depend on it unless it adds functionality you really need.
If I did apply that particular patch, I would want some rework because
it currently obfuscates the match_driver logic. There's no clue when
reading the code what -1/0/1 mean.
So do you prefer enum's? - If so I can make a change for that.
Apparently here you *do* want the "-1 means the PCI core will never
set match_driver to 1" functionality, so maybe you do depend on it.
We depend on the patch because we want that ability to never set
match_driver,
for SRIOV on PowerVM.
If that's the case, how to you ever bind a driver to these VFs? The
changelog says you don't want VF drivers to load *immediately*, so I
assume you do want them to load eventually.
The VF's that get dynamically created within the configure SR-IOV call,
on the
Pseries Platform, wont be matched with a driver. - We do not want it to
match.
The Power Hypervisor will load the VFs. The VF's will get assigned(by
the user)
via the HMC or Novalink in this environment which will then trigger PHYP
to load the VF device node to the device tree.
-Bryant
Re: [PATCH v1 1/3] powerpc/kernel: Split up pci_bus_add_device
On 9/21/17 3:43 PM, Bjorn Helgaas wrote:
On Mon, Sep 18, 2017 at 02:26:49PM -0500, Bryant G. Ly wrote:
When enabling SR-IOV one might want to have their
own version of starting device drivers for the VFs.
This patch allows for SR-IOV callers to use
pci_bus_add_virtfn_device instead of generic
pci_bus_add_device.
When enabling SR-IOV in PSeries architecture the
dynamic VFs created within the sriov_configure sysfs call
will not load the device driver as firmware will load
the device node when the VF device is assigned to the
logical partition. So we needed a way to overwrite the
way device driver matching is done for virtual functions
on powervm.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
drivers/pci/bus.c | 51 ++-
drivers/pci/iov.c | 2 +-
include/linux/pci.h | 3 +++
3 files changed, 46 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index bc56cf19afd3..86daf62c4048 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -302,16 +302,9 @@ void __weak pcibios_resource_survey_bus(struct pci_bus
*bus) { }
void __weak pcibios_bus_add_device(struct pci_dev *pdev) { }
-/**
- * pci_bus_add_device - start driver for a single device
- * @dev: device to add
- *
- * This adds add sysfs entries and start device drivers
- */
-void pci_bus_add_device(struct pci_dev *dev)
-{
- int retval;
+void pci_bus_add_sysfs_entries(struct pci_dev *dev)
+{
/*
* Can not put in pci_device_add yet because resources
* are not assigned yet for some devices.
@@ -321,6 +314,11 @@ void pci_bus_add_device(struct pci_dev *dev)
pci_create_sysfs_dev_files(dev);
pci_proc_attach_device(dev);
pci_bridge_d3_update(dev);
+}
+
+void pci_bus_match_device_driver(struct pci_dev *dev)
+{
+ int retval;
dev->match_driver = true;
retval = device_attach(>dev);
@@ -333,6 +331,41 @@ void pci_bus_add_device(struct pci_dev *dev)
dev->is_added = 1;
}
+
+#ifdef CONFIG_PCI_IOV
+void __weak pci_bus_match_virtfn_driver(struct pci_dev *dev)
+{
+ pci_bus_match_device_driver(dev);
+}
+
+/**
+ * pci_bus_add_virtfn_device - start driver for a virtual function device
+ * @dev: device to add
+ *
+ * This adds add sysfs entries and start device drivers for
+ * virtual function devices
+ *
+ */
+void pci_bus_add_virtfn_device(struct pci_dev *pdev)
+{
+ pci_bus_add_sysfs_entries(pdev);
+ pci_bus_match_virtfn_driver(pdev);
+}
+EXPORT_SYMBOL_GPL(pci_bus_add_virtfn_device);
Is there any way we can avoid adding this new interface? I don't
really want a new global exported symbol just to support this highly
platform-specific functionality.
If we could figure out a way to set dev->match_driver to false instead
of true in pci_bus_add_device(), it *looks* like that would be enough
to accomplish what you need. Maybe that could be done via a pcibios
hook or a new function pointer in struct pci_host_bridge, similar to
the swizzle_irq pointer?
Yes! I will remove the first two patches in the series and only utilize the
third
patch but addingpdev->match_driver = -1; under pseries_pcibios_bus_add_device, which is
a new function I added for machine dependent bus add of a device. The
patch will then require https://patchwork.kernel.org/patch/9882915/
which basically allows for match_driver state to not get changed later
under the pci_bus_match due to allowing for the match_driver to be a int
vs a bool. I will push up another patch removing the first two patches
and adding that extra line. -Bryant
+#endif
+
+/**
+ * pci_bus_add_device - start driver for a single device
+ * @dev: device to add
+ *
+ * This adds add sysfs entries and start device drivers
+ */
+void pci_bus_add_device(struct pci_dev *dev)
+{
+ pci_bus_add_sysfs_entries(dev);
+ pci_bus_match_device_driver(dev);
+}
+
EXPORT_SYMBOL_GPL(pci_bus_add_device);
/**
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ac41c8be9200..16cc72545847 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -162,7 +162,7 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id, int
reset)
pci_device_add(virtfn, virtfn->bus);
- pci_bus_add_device(virtfn);
+ pci_bus_add_virtfn_device(virtfn);
sprintf(buf, "virtfn%u", id);
rc = sysfs_create_link(>dev.kobj, >dev.kobj, buf);
if (rc)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index f68c58a93dd0..39f5c0b4bf23 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -911,6 +911,9 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus,
int devfn);
void pci_device_add(struct pci_dev *dev, struct pci_bus *bus);
unsigned int pci_scan_child_bus(struct pci_bus *bus);
void pci_bus_add_device(struct pci_dev *dev);
+#ifdef CONFIG_PCI_IOV
+void pci_bus_add_virt
[PATCH v3 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device
This patch adds the machine dependent call for
pcibios_bus_add_device, since the previous patch
separated the calls out between the PowerNV and PowerVM.
The difference here is that for the PowerVM environment
we do not want match_driver set because in this environment
we do not want the VF device drivers to load immediately, due to
firmware loading the device node when VF device is assigned to the
logical partition.
This patch will depend on the patch linked below, which is under
review.
https://patchwork.kernel.org/patch/9882915/
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 24
1 file changed, 24 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 6b812ad990e4..45946ee90985 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -64,6 +64,27 @@ static unsigned char slot_errbuf[RTAS_ERROR_LOG_MAX];
static DEFINE_SPINLOCK(slot_errbuf_lock);
static int eeh_error_buf_size;
+void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdev->is_virtfn)
+ return;
+
+ pdn->device_id = pdev->device;
+ pdn->vendor_id = pdev->vendor;
+ pdn->class_code = pdev->class;
+
+ /*
+* The following operations will fail if VF's sysfs files
+* aren't created or its resources aren't finalized.
+*/
+ eeh_add_device_early(pdn);
+ eeh_add_device_late(pdev);
+ eeh_sysfs_add_device(pdev);
+ pdev->match_driver = -1;
+}
+
/**
* pseries_eeh_init - EEH platform dependent initialization
*
@@ -120,6 +141,9 @@ static int pseries_eeh_init(void)
/* Set EEH probe mode */
eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
+ /* Set EEH machine dependent code */
+ ppc_md.pcibios_bus_add_device = pseries_pcibios_bus_add_device;
+
return 0;
}
--
2.11.0 (Apple Git-81)
[PATCH v3 0/2] Prepartion for SR-IOV PowerVM Enablement
This patch series is to prepare for enabling SR-IOV on pseries. It separates the calls to be machine dependent and does not change any current functionality. The patch linked below which is currently in review is a dependency: https://patchwork.kernel.org/patch/9882915/ v1 - Initial patch v2 - Addressed Bjorn's comment on creating a highly platform dependent global exported symbol. v3 - Based patch off linux-ppc/master Bryant G. Ly (2): powerpc/kernel: Separate SR-IOV Calls pseries/eeh: Add Pseries pcibios_bus_add_device arch/powerpc/include/asm/machdep.h | 7 ++ arch/powerpc/include/asm/pci-bridge.h| 4 +--- arch/powerpc/kernel/eeh_driver.c | 4 ++-- arch/powerpc/kernel/pci-common.c | 23 +++ arch/powerpc/kernel/pci_dn.c | 6 - arch/powerpc/platforms/powernv/eeh-powernv.c | 33 ++-- arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- arch/powerpc/platforms/pseries/eeh_pseries.c | 24 8 files changed, 78 insertions(+), 29 deletions(-) -- 2.11.0 (Apple Git-81)
[PATCH v3 1/2] powerpc/kernel: Separate SR-IOV Calls
SR-IOV can now be enabled in PowerNV platforms and Pseries
platforms. Therefore, the appropriate calls were moved to
machine dependent code instead of definition at compile time.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/include/asm/machdep.h | 7 ++
arch/powerpc/include/asm/pci-bridge.h| 4 +---
arch/powerpc/kernel/eeh_driver.c | 4 ++--
arch/powerpc/kernel/pci-common.c | 23 +++
arch/powerpc/kernel/pci_dn.c | 6 -
arch/powerpc/platforms/powernv/eeh-powernv.c | 33 ++--
arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++--
7 files changed, 54 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/machdep.h
b/arch/powerpc/include/asm/machdep.h
index 73b92017b6d7..20f68d36af8c 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -172,11 +172,18 @@ struct machdep_calls {
/* Called after scan and before resource survey */
void (*pcibios_fixup_phb)(struct pci_controller *hose);
+ /* Called after device has been added to bus and
+* before sysfs has been created
+*/
+ void (*pcibios_bus_add_device)(struct pci_dev *pdev);
+
resource_size_t (*pcibios_default_alignment)(void);
#ifdef CONFIG_PCI_IOV
void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int
resno);
+ int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
+ int (*pcibios_sriov_disable)(struct pci_dev *pdev);
#endif /* CONFIG_PCI_IOV */
/* Called to shutdown machine specific hardware not already controlled
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 0b8aa1fe2d5f..323628ca4d6d 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -203,10 +203,9 @@ struct pci_dn {
struct eeh_dev *edev; /* eeh device */
#endif
#define IODA_INVALID_PE0x
-#ifdef CONFIG_PPC_POWERNV
unsigned int pe_number;
- int vf_index; /* VF index in the PF */
#ifdef CONFIG_PCI_IOV
+ int vf_index; /* VF index in the PF */
u16 vfs_expanded; /* number of VFs IOV BAR expanded */
u16 num_vfs;/* number of VFs enabled*/
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
@@ -215,7 +214,6 @@ struct pci_dn {
int (*m64_map)[PCI_SRIOV_NUM_BARS];
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
-#endif
struct list_head child_list;
struct list_head list;
};
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 8b840191df59..f2d1b369974d 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -440,7 +440,7 @@ static void *eeh_add_virt_device(void *data, void *userdata)
return NULL;
}
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
pci_iov_add_virtfn(edev->physfn, pdn->vf_index, 0);
#endif
return NULL;
@@ -496,7 +496,7 @@ static void *eeh_rmv_device(void *data, void *userdata)
(*removed)++;
if (edev->physfn) {
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
struct pci_dn *pdn = eeh_dev_to_pdn(edev);
pci_iov_remove_virtfn(edev->physfn, pdn->vf_index, 0);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 02831a396419..d45b956d2e3a 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -249,8 +249,31 @@ resource_size_t pcibios_iov_resource_alignment(struct
pci_dev *pdev, int resno)
return pci_iov_resource_size(pdev, resno);
}
+
+int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ if (ppc_md.pcibios_sriov_enable)
+ return ppc_md.pcibios_sriov_enable(pdev, num_vfs);
+
+ return 0;
+}
+
+int pcibios_sriov_disable(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_sriov_disable)
+ return ppc_md.pcibios_sriov_disable(pdev);
+
+ return 0;
+}
+
#endif /* CONFIG_PCI_IOV */
+void pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_bus_add_device)
+ ppc_md.pcibios_bus_add_device(pdev);
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 0e395afbf0f4..ab147a1909c8 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -156,10 +156,8 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn
*parent,
pdn-&g
[PATCH v2 2/2] pseries/eeh: Add Pseries pcibios_bus_add_device
This patch adds the machine dependent call for
pcibios_bus_add_device, since the previous patch
separated the calls out between the PowerNV and PowerVM.
The difference here is that for the PowerVM environment
we do not want match_driver set because in this environment
we do not want the VF device drivers to load immediately, due to
firmware loading the device node when VF device is assigned to the
logical partition.
This patch will depend on the patch linked below, which is under
review.
https://patchwork.kernel.org/patch/9882915/
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/eeh_pseries.c | 24
1 file changed, 24 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/eeh_pseries.c
b/arch/powerpc/platforms/pseries/eeh_pseries.c
index 1eef46d9cf30..ac72011a045c 100644
--- a/arch/powerpc/platforms/pseries/eeh_pseries.c
+++ b/arch/powerpc/platforms/pseries/eeh_pseries.c
@@ -64,6 +64,27 @@ static unsigned char slot_errbuf[RTAS_ERROR_LOG_MAX];
static DEFINE_SPINLOCK(slot_errbuf_lock);
static int eeh_error_buf_size;
+void pseries_pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ struct pci_dn *pdn = pci_get_pdn(pdev);
+
+ if (!pdev->is_virtfn)
+ return;
+
+ pdn->device_id = pdev->device;
+ pdn->vendor_id = pdev->vendor;
+ pdn->class_code = pdev->class;
+
+ /*
+* The following operations will fail if VF's sysfs files
+* aren't created or its resources aren't finalized.
+*/
+ eeh_add_device_early(pdn);
+ eeh_add_device_late(pdev);
+ eeh_sysfs_add_device(pdev);
+ pdev->match_driver = -1;
+}
+
/**
* pseries_eeh_init - EEH platform dependent initialization
*
@@ -120,6 +141,9 @@ static int pseries_eeh_init(void)
/* Set EEH probe mode */
eeh_add_flag(EEH_PROBE_MODE_DEVTREE | EEH_ENABLE_IO_FOR_LOG);
+ /* Set EEH machine dependent code */
+ ppc_md.pcibios_bus_add_device = pseries_pcibios_bus_add_device;
+
return 0;
}
--
2.11.0 (Apple Git-81)
[PATCH v2 1/2] powerpc/kernel: Separate SR-IOV Calls
SR-IOV can now be enabled in PowerNV platforms and Pseries
platforms. Therefore, the appropriate calls were moved to
machine dependent code instead of definition at compile time.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/include/asm/machdep.h | 7 ++
arch/powerpc/include/asm/pci-bridge.h| 4 +---
arch/powerpc/kernel/eeh_driver.c | 4 ++--
arch/powerpc/kernel/pci-common.c | 23 +++
arch/powerpc/kernel/pci_dn.c | 6 -
arch/powerpc/platforms/powernv/eeh-powernv.c | 34 +++-
arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++--
7 files changed, 55 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/machdep.h
b/arch/powerpc/include/asm/machdep.h
index f90b22c722e1..57b6608bb4fb 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -173,11 +173,18 @@ struct machdep_calls {
/* Called after scan and before resource survey */
void (*pcibios_fixup_phb)(struct pci_controller *hose);
+ /* Called after device has been added to bus and
+* before sysfs has been created
+*/
+ void (*pcibios_bus_add_device)(struct pci_dev *pdev);
+
resource_size_t (*pcibios_default_alignment)(void);
#ifdef CONFIG_PCI_IOV
void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int
resno);
+ int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
+ int (*pcibios_sriov_disable)(struct pci_dev *pdev);
#endif /* CONFIG_PCI_IOV */
/* Called to shutdown machine specific hardware not already controlled
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 56c67d3f0108..e5bf0f15fc4d 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -204,10 +204,9 @@ struct pci_dn {
struct eeh_dev *edev; /* eeh device */
#endif
#define IODA_INVALID_PE0x
-#ifdef CONFIG_PPC_POWERNV
unsigned int pe_number;
- int vf_index; /* VF index in the PF */
#ifdef CONFIG_PCI_IOV
+ int vf_index; /* VF index in the PF */
u16 vfs_expanded; /* number of VFs IOV BAR expanded */
u16 num_vfs;/* number of VFs enabled*/
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
@@ -216,7 +215,6 @@ struct pci_dn {
int (*m64_map)[PCI_SRIOV_NUM_BARS];
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
-#endif
struct list_head child_list;
struct list_head list;
};
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index c405c79e50cd..ee5805926e77 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -440,7 +440,7 @@ static void *eeh_add_virt_device(void *data, void *userdata)
return NULL;
}
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
pci_iov_add_virtfn(edev->physfn, pdn->vf_index, 0);
#endif
return NULL;
@@ -496,7 +496,7 @@ static void *eeh_rmv_device(void *data, void *userdata)
(*removed)++;
if (edev->physfn) {
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
struct pci_dn *pdn = eeh_dev_to_pdn(edev);
pci_iov_remove_virtfn(edev->physfn, pdn->vf_index, 0);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 6fae7654017e..786040138aec 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -249,8 +249,31 @@ resource_size_t pcibios_iov_resource_alignment(struct
pci_dev *pdev, int resno)
return pci_iov_resource_size(pdev, resno);
}
+
+int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ if (ppc_md.pcibios_sriov_enable)
+ return ppc_md.pcibios_sriov_enable(pdev, num_vfs);
+
+ return 0;
+}
+
+int pcibios_sriov_disable(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_sriov_disable)
+ return ppc_md.pcibios_sriov_disable(pdev);
+
+ return 0;
+}
+
#endif /* CONFIG_PCI_IOV */
+void pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_bus_add_device)
+ ppc_md.pcibios_bus_add_device(pdev);
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 592693437070..52659bb7450f 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -159,10 +159,8 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn
*parent,
pdn-&g
[PATCH v2 0/2] Prepartion for SR-IOV PowerVM Enablement
This patch series is to prepare for enabling SR-IOV on pseries. It separates the calls to be machine dependent and does not change any current functionality. This patch which is currently in review is a dependency: https://patchwork.kernel.org/patch/9882915/ Bryant G. Ly (2): powerpc/kernel: Separate SR-IOV Calls pseries/eeh: Add Pseries pcibios_bus_add_device arch/powerpc/include/asm/machdep.h | 7 ++ arch/powerpc/include/asm/pci-bridge.h| 4 +--- arch/powerpc/kernel/eeh_driver.c | 4 ++-- arch/powerpc/kernel/pci-common.c | 23 +++ arch/powerpc/kernel/pci_dn.c | 6 - arch/powerpc/platforms/powernv/eeh-powernv.c | 34 +++- arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++-- arch/powerpc/platforms/pseries/eeh_pseries.c | 24 8 files changed, 79 insertions(+), 29 deletions(-) -- 2.11.0 (Apple Git-81)
[PATCH v1 3/3] powerpc/kernel: Separate SR-IOV Calls
SR-IOV can now be enabled in PowerNV platforms and Pseries
platforms. Therefore, the appropriate calls were moved to
machine dependent code instead of definition at compile time.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/include/asm/machdep.h | 7 ++
arch/powerpc/include/asm/pci-bridge.h| 4 +---
arch/powerpc/kernel/eeh_driver.c | 4 ++--
arch/powerpc/kernel/pci-common.c | 23 +++
arch/powerpc/kernel/pci_dn.c | 6 -
arch/powerpc/platforms/powernv/eeh-powernv.c | 34 +++-
arch/powerpc/platforms/powernv/pci-ioda.c| 6 +++--
7 files changed, 55 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/include/asm/machdep.h
b/arch/powerpc/include/asm/machdep.h
index 73b92017b6d7..20f68d36af8c 100644
--- a/arch/powerpc/include/asm/machdep.h
+++ b/arch/powerpc/include/asm/machdep.h
@@ -172,11 +172,18 @@ struct machdep_calls {
/* Called after scan and before resource survey */
void (*pcibios_fixup_phb)(struct pci_controller *hose);
+ /* Called after device has been added to bus and
+* before sysfs has been created
+*/
+ void (*pcibios_bus_add_device)(struct pci_dev *pdev);
+
resource_size_t (*pcibios_default_alignment)(void);
#ifdef CONFIG_PCI_IOV
void (*pcibios_fixup_sriov)(struct pci_dev *pdev);
resource_size_t (*pcibios_iov_resource_alignment)(struct pci_dev *, int
resno);
+ int (*pcibios_sriov_enable)(struct pci_dev *pdev, u16 num_vfs);
+ int (*pcibios_sriov_disable)(struct pci_dev *pdev);
#endif /* CONFIG_PCI_IOV */
/* Called to shutdown machine specific hardware not already controlled
diff --git a/arch/powerpc/include/asm/pci-bridge.h
b/arch/powerpc/include/asm/pci-bridge.h
index 0b8aa1fe2d5f..323628ca4d6d 100644
--- a/arch/powerpc/include/asm/pci-bridge.h
+++ b/arch/powerpc/include/asm/pci-bridge.h
@@ -203,10 +203,9 @@ struct pci_dn {
struct eeh_dev *edev; /* eeh device */
#endif
#define IODA_INVALID_PE0x
-#ifdef CONFIG_PPC_POWERNV
unsigned int pe_number;
- int vf_index; /* VF index in the PF */
#ifdef CONFIG_PCI_IOV
+ int vf_index; /* VF index in the PF */
u16 vfs_expanded; /* number of VFs IOV BAR expanded */
u16 num_vfs;/* number of VFs enabled*/
unsigned int *pe_num_map; /* PE# for the first VF PE or array */
@@ -215,7 +214,6 @@ struct pci_dn {
int (*m64_map)[PCI_SRIOV_NUM_BARS];
#endif /* CONFIG_PCI_IOV */
int mps;/* Maximum Payload Size */
-#endif
struct list_head child_list;
struct list_head list;
};
diff --git a/arch/powerpc/kernel/eeh_driver.c b/arch/powerpc/kernel/eeh_driver.c
index 8b840191df59..f2d1b369974d 100644
--- a/arch/powerpc/kernel/eeh_driver.c
+++ b/arch/powerpc/kernel/eeh_driver.c
@@ -440,7 +440,7 @@ static void *eeh_add_virt_device(void *data, void *userdata)
return NULL;
}
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
pci_iov_add_virtfn(edev->physfn, pdn->vf_index, 0);
#endif
return NULL;
@@ -496,7 +496,7 @@ static void *eeh_rmv_device(void *data, void *userdata)
(*removed)++;
if (edev->physfn) {
-#ifdef CONFIG_PPC_POWERNV
+#ifdef CONFIG_PCI_IOV
struct pci_dn *pdn = eeh_dev_to_pdn(edev);
pci_iov_remove_virtfn(edev->physfn, pdn->vf_index, 0);
diff --git a/arch/powerpc/kernel/pci-common.c b/arch/powerpc/kernel/pci-common.c
index 02831a396419..d45b956d2e3a 100644
--- a/arch/powerpc/kernel/pci-common.c
+++ b/arch/powerpc/kernel/pci-common.c
@@ -249,8 +249,31 @@ resource_size_t pcibios_iov_resource_alignment(struct
pci_dev *pdev, int resno)
return pci_iov_resource_size(pdev, resno);
}
+
+int pcibios_sriov_enable(struct pci_dev *pdev, u16 num_vfs)
+{
+ if (ppc_md.pcibios_sriov_enable)
+ return ppc_md.pcibios_sriov_enable(pdev, num_vfs);
+
+ return 0;
+}
+
+int pcibios_sriov_disable(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_sriov_disable)
+ return ppc_md.pcibios_sriov_disable(pdev);
+
+ return 0;
+}
+
#endif /* CONFIG_PCI_IOV */
+void pcibios_bus_add_device(struct pci_dev *pdev)
+{
+ if (ppc_md.pcibios_bus_add_device)
+ ppc_md.pcibios_bus_add_device(pdev);
+}
+
static resource_size_t pcibios_io_size(const struct pci_controller *hose)
{
#ifdef CONFIG_PPC64
diff --git a/arch/powerpc/kernel/pci_dn.c b/arch/powerpc/kernel/pci_dn.c
index 0e395afbf0f4..ab147a1909c8 100644
--- a/arch/powerpc/kernel/pci_dn.c
+++ b/arch/powerpc/kernel/pci_dn.c
@@ -156,10 +156,8 @@ static struct pci_dn *add_one_dev_pci_data(struct pci_dn
*parent,
pdn-&g
[PATCH v1 2/3] pseries: Override pci_bus_match_virtfn_driver
For Powervm SR-IOV (Pseries) enablement we dont want to match
the virtual function's device drivers since firmware
plans to load the device node in the device tree
dynamically when Novalink assigns the VF to a partition.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
arch/powerpc/platforms/pseries/pci.c | 17 +
1 file changed, 17 insertions(+)
diff --git a/arch/powerpc/platforms/pseries/pci.c
b/arch/powerpc/platforms/pseries/pci.c
index 09eba5a9929a..15d5145a622d 100644
--- a/arch/powerpc/platforms/pseries/pci.c
+++ b/arch/powerpc/platforms/pseries/pci.c
@@ -58,6 +58,23 @@ void pcibios_name_device(struct pci_dev *dev)
DECLARE_PCI_FIXUP_HEADER(PCI_ANY_ID, PCI_ANY_ID, pcibios_name_device);
#endif
+#ifdef CONFIG_PCI_IOV
+void pci_bus_match_virtfn_driver(struct pci_dev *dev)
+{
+ /*
+* Per PSeries SR-IOV requirement there is no need to
+* match virtual function device driver as firmware
+* will load the device node in the device tree dynamically.
+* Since there is no matching of device driver there is
+* no failure when attaching driver, therefore there is no
+* need to remove sysfs file. Furthermore, the VF platform
+* management still needs to exist in sysfs files to be used
+* by management.
+*/
+ dev->is_added = 1;
+}
+#endif
+
static void __init pSeries_request_regions(void)
{
if (!isa_io_base)
--
2.11.0 (Apple Git-81)
[PATCH v1 1/3] powerpc/kernel: Split up pci_bus_add_device
When enabling SR-IOV one might want to have their
own version of starting device drivers for the VFs.
This patch allows for SR-IOV callers to use
pci_bus_add_virtfn_device instead of generic
pci_bus_add_device.
When enabling SR-IOV in PSeries architecture the
dynamic VFs created within the sriov_configure sysfs call
will not load the device driver as firmware will load
the device node when the VF device is assigned to the
logical partition. So we needed a way to overwrite the
way device driver matching is done for virtual functions
on powervm.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalv...@us.ibm.com>
---
drivers/pci/bus.c | 51 ++-
drivers/pci/iov.c | 2 +-
include/linux/pci.h | 3 +++
3 files changed, 46 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c
index bc56cf19afd3..86daf62c4048 100644
--- a/drivers/pci/bus.c
+++ b/drivers/pci/bus.c
@@ -302,16 +302,9 @@ void __weak pcibios_resource_survey_bus(struct pci_bus
*bus) { }
void __weak pcibios_bus_add_device(struct pci_dev *pdev) { }
-/**
- * pci_bus_add_device - start driver for a single device
- * @dev: device to add
- *
- * This adds add sysfs entries and start device drivers
- */
-void pci_bus_add_device(struct pci_dev *dev)
-{
- int retval;
+void pci_bus_add_sysfs_entries(struct pci_dev *dev)
+{
/*
* Can not put in pci_device_add yet because resources
* are not assigned yet for some devices.
@@ -321,6 +314,11 @@ void pci_bus_add_device(struct pci_dev *dev)
pci_create_sysfs_dev_files(dev);
pci_proc_attach_device(dev);
pci_bridge_d3_update(dev);
+}
+
+void pci_bus_match_device_driver(struct pci_dev *dev)
+{
+ int retval;
dev->match_driver = true;
retval = device_attach(>dev);
@@ -333,6 +331,41 @@ void pci_bus_add_device(struct pci_dev *dev)
dev->is_added = 1;
}
+
+#ifdef CONFIG_PCI_IOV
+void __weak pci_bus_match_virtfn_driver(struct pci_dev *dev)
+{
+ pci_bus_match_device_driver(dev);
+}
+
+/**
+ * pci_bus_add_virtfn_device - start driver for a virtual function device
+ * @dev: device to add
+ *
+ * This adds add sysfs entries and start device drivers for
+ * virtual function devices
+ *
+ */
+void pci_bus_add_virtfn_device(struct pci_dev *pdev)
+{
+ pci_bus_add_sysfs_entries(pdev);
+ pci_bus_match_virtfn_driver(pdev);
+}
+EXPORT_SYMBOL_GPL(pci_bus_add_virtfn_device);
+#endif
+
+/**
+ * pci_bus_add_device - start driver for a single device
+ * @dev: device to add
+ *
+ * This adds add sysfs entries and start device drivers
+ */
+void pci_bus_add_device(struct pci_dev *dev)
+{
+ pci_bus_add_sysfs_entries(dev);
+ pci_bus_match_device_driver(dev);
+}
+
EXPORT_SYMBOL_GPL(pci_bus_add_device);
/**
diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
index ac41c8be9200..16cc72545847 100644
--- a/drivers/pci/iov.c
+++ b/drivers/pci/iov.c
@@ -162,7 +162,7 @@ int pci_iov_add_virtfn(struct pci_dev *dev, int id, int
reset)
pci_device_add(virtfn, virtfn->bus);
- pci_bus_add_device(virtfn);
+ pci_bus_add_virtfn_device(virtfn);
sprintf(buf, "virtfn%u", id);
rc = sysfs_create_link(>dev.kobj, >dev.kobj, buf);
if (rc)
diff --git a/include/linux/pci.h b/include/linux/pci.h
index f68c58a93dd0..39f5c0b4bf23 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -911,6 +911,9 @@ struct pci_dev *pci_scan_single_device(struct pci_bus *bus,
int devfn);
void pci_device_add(struct pci_dev *dev, struct pci_bus *bus);
unsigned int pci_scan_child_bus(struct pci_bus *bus);
void pci_bus_add_device(struct pci_dev *dev);
+#ifdef CONFIG_PCI_IOV
+void pci_bus_add_virtfn_device(struct pci_dev *dev);
+#endif
void pci_read_bridge_bases(struct pci_bus *child);
struct resource *pci_find_parent_resource(const struct pci_dev *dev,
struct resource *res);
--
2.11.0 (Apple Git-81)
[PATCH v1 0/3] Prepartion for SR-IOV PowerVM Enablement
This patch series is to prepare for enabling SR-IOV on pseries. It separates the calls to be machine dependent and does not change any current functionality. Bryant G. Ly (3): powerpc/kernel: Split up pci_bus_add_device pseries: Override pci_bus_match_virtfn_driver powerpc/kernel: Separate SR-IOV Calls arch/powerpc/include/asm/machdep.h | 7 arch/powerpc/include/asm/pci-bridge.h| 4 +-- arch/powerpc/kernel/eeh_driver.c | 4 +-- arch/powerpc/kernel/pci-common.c | 23 + arch/powerpc/kernel/pci_dn.c | 6 arch/powerpc/platforms/powernv/eeh-powernv.c | 34 ++- arch/powerpc/platforms/powernv/pci-ioda.c| 6 ++-- arch/powerpc/platforms/pseries/pci.c | 17 ++ drivers/pci/bus.c| 51 +++- drivers/pci/iov.c| 2 +- include/linux/pci.h | 3 ++ 11 files changed, 118 insertions(+), 39 deletions(-) -- 2.11.0 (Apple Git-81)
Re: [PATCH] powerpc/kernel: Change retrieval of pci_dn
On 8/29/17 1:20 AM, Sam Bobroff wrote:
On Mon, Aug 28, 2017 at 11:05:03AM -0500, Bryant G. Ly wrote:
For a PCI device it's pci_dn can be retrieved from
pdev->dev.archdata.firmware_data, PCI_DN(devnode), or parent's list.
Thus, we should just use the generic function pci_get_pdn_by_devfn
to get the pci_dn.
Signed-off-by: Bryant G. Ly <b...@us.ibm.com>
Reviewed-by: Sam Bobroff <sam.bobr...@au1.ibm.com>
I don't know this area but I tested it using a patched kernel with the
old and new code together. My test kernel booted fine (in QEMU+KVM) and
I saw 26 reads and 4 writes, all of which got the same value with either
code block.
I also checked that the error result in the "not found" case is the same
as well, which it is, because rtas_{read,write}_config() will return
PCIBIOS_DEVICE_NOT_FOUND if given a NULL pdn.
So, looks good to me.
Cheers,
Sam.
Thanks for the review Sam!
Re: [PATCH] powerpc/kernel: Change retrieval of pci_dn
On 8/29/17 1:33 AM, Michael Ellerman wrote: "Bryant G. Ly" <bryan...@linux.vnet.ibm.com> writes: For a PCI device it's pci_dn can be retrieved from pdev->dev.archdata.firmware_data, PCI_DN(devnode), or parent's list. Thus, we should just use the generic function pci_get_pdn_by_devfn to get the pci_dn. Signed-off-by: Bryant G. Ly <b...@us.ibm.com> Minor issue, it's preferable if the email in your Signed-off-by matches the email you send patches from. cheers Hi Michael, Thanks for the review. I apologize for the email's not matching, I switch between the two frequently throughout the day for internal gerrit commits and Linux patches. I have addressed all your comments in the new patch that I had just put up. Also, I have tested it with mellanox cx4 cards on P8 systems. I'd also like to let you know that I am working on patches to enable SRIOV on power and would like your feedback on design, which I will send in a private email. -Bryant
[PATCH] powerpc/kernel: Change retrieval of pci_dn
For a PCI device it's pci_dn can be retrieved from
pdev->dev.archdata.firmware_data, PCI_DN(devnode), or parent's list.
Thus, we should just use the existing function pci_get_pdn_by_devfn
to get the pci_dn.
Signed-off-by: Bryant G. Ly <bryan...@linux.vnet.ibm.com>
Reviewed-by: Sam Bobroff <sam.bobr...@au1.ibm.com>
---
arch/powerpc/kernel/rtas_pci.c | 33 -
1 file changed, 4 insertions(+), 29 deletions(-)
diff --git a/arch/powerpc/kernel/rtas_pci.c b/arch/powerpc/kernel/rtas_pci.c
index 73f1934..c2b148b 100644
--- a/arch/powerpc/kernel/rtas_pci.c
+++ b/arch/powerpc/kernel/rtas_pci.c
@@ -91,26 +91,14 @@ static int rtas_pci_read_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 *val)
{
- struct device_node *busdn, *dn;
struct pci_dn *pdn;
- bool found = false;
int ret;
- /* Search only direct children of the bus */
*val = 0x;
- busdn = pci_bus_to_OF_node(bus);
- for (dn = busdn->child; dn; dn = dn->sibling) {
- pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn)) {
- found = true;
- break;
- }
- }
- if (!found)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ pdn = pci_get_pdn_by_devfn(bus, devfn);
+ /* Validity of pdn is checked in here */
ret = rtas_read_config(pdn, where, size, val);
if (*val == EEH_IO_ERROR_VALUE(size) &&
eeh_dev_check_failure(pdn_to_eeh_dev(pdn)))
@@ -153,24 +141,11 @@ static int rtas_pci_write_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 val)
{
- struct device_node *busdn, *dn;
struct pci_dn *pdn;
- bool found = false;
-
- /* Search only direct children of the bus */
- busdn = pci_bus_to_OF_node(bus);
- for (dn = busdn->child; dn; dn = dn->sibling) {
- pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn)) {
- found = true;
- break;
- }
- }
- if (!found)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ pdn = pci_get_pdn_by_devfn(bus, devfn);
+ /* Validity of pdn is checked in here. */
return rtas_write_config(pdn, where, size, val);
}
--
2.5.4 (Apple Git-61)
[PATCH] powerpc/kernel: Change retrieval of pci_dn
For a PCI device it's pci_dn can be retrieved from
pdev->dev.archdata.firmware_data, PCI_DN(devnode), or parent's list.
Thus, we should just use the generic function pci_get_pdn_by_devfn
to get the pci_dn.
Signed-off-by: Bryant G. Ly <b...@us.ibm.com>
---
arch/powerpc/kernel/rtas_pci.c | 30 ++
1 file changed, 2 insertions(+), 28 deletions(-)
diff --git a/arch/powerpc/kernel/rtas_pci.c b/arch/powerpc/kernel/rtas_pci.c
index 73f1934..c21e6c5 100644
--- a/arch/powerpc/kernel/rtas_pci.c
+++ b/arch/powerpc/kernel/rtas_pci.c
@@ -91,25 +91,13 @@ static int rtas_pci_read_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 *val)
{
- struct device_node *busdn, *dn;
struct pci_dn *pdn;
- bool found = false;
int ret;
/* Search only direct children of the bus */
*val = 0x;
- busdn = pci_bus_to_OF_node(bus);
- for (dn = busdn->child; dn; dn = dn->sibling) {
- pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn)) {
- found = true;
- break;
- }
- }
- if (!found)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ pdn = pci_get_pdn_by_devfn(bus, devfn);
ret = rtas_read_config(pdn, where, size, val);
if (*val == EEH_IO_ERROR_VALUE(size) &&
@@ -153,23 +141,9 @@ static int rtas_pci_write_config(struct pci_bus *bus,
unsigned int devfn,
int where, int size, u32 val)
{
- struct device_node *busdn, *dn;
struct pci_dn *pdn;
- bool found = false;
-
- /* Search only direct children of the bus */
- busdn = pci_bus_to_OF_node(bus);
- for (dn = busdn->child; dn; dn = dn->sibling) {
- pdn = PCI_DN(dn);
- if (pdn && pdn->devfn == devfn
- && of_device_is_available(dn)) {
- found = true;
- break;
- }
- }
- if (!found)
- return PCIBIOS_DEVICE_NOT_FOUND;
+ pdn = pci_get_pdn_by_devfn(bus, devfn);
return rtas_write_config(pdn, where, size, val);
}
--
2.5.4 (Apple Git-61)
